2026-02-20T22:35:50.242 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0
2026-02-20T22:35:50.256 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T22:35:50.333 INFO:teuthology.run:Config:
  archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616
  branch: wip-pdonnell-testing-20260219.182737-tentacle
  description: orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start
    2-services/mirror 3-final}
  email: yweinste@redhat.com
  first_in_suite: false
  flavor: default
  job_id: '61616'
  kernel:
    branch: distro
    kdb: true
    sha1: distro
  last_in_suite: false
  machine_type: trial
  name: yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial
  no_nested_subset: false
  openstack:
  - volumes:
      count: 4
      size: 10
  os_type: centos
  os_version: 9.stream
  overrides:
    admin_socket:
      branch: wip-pdonnell-testing-20260219.182737-tentacle
    ceph:
      conf:
        mgr:
          debug mgr: 20
          debug ms: 1
        mon:
          debug mon: 20
          debug ms: 1
          debug paxos: 20
        osd:
          debug ms: 1
          debug osd: 20
          osd shutdown pgref assert: true
      flavor: default
      log-ignorelist:
      - \(MDS_ALL_DOWN\)
      - \(MDS_UP_LESS_THAN_MAX\)
      - CEPHADM_DAEMON_PLACE_FAIL
      - CEPHADM_FAILED_DAEMON
      log-only-match:
      - CEPHADM_
      sha1: f7870454149f22574086dd2ff55d050585136531
    ceph-deploy:
      conf:
        client:
          log file: /var/log/ceph/ceph-$name.$pid.log
        mon: {}
    install:
      ceph:
        flavor: default
        sha1: f7870454149f22574086dd2ff55d050585136531
    selinux:
      allowlist:
      - scontext=system_u:system_r:logrotate_t:s0
      - scontext=system_u:system_r:getty_t:s0
    workunit:
      branch: wip-pdonnell-testing-20260219.182737-tentacle
      sha1: f7870454149f22574086dd2ff55d050585136531
  owner: scheduled_yuriw@soko04.front.sepia.ceph.com
  priority: 69
  repo: https://github.com/ceph/ceph-ci.git
  roles:
  - - host.a
    - client.0
  - - host.b
    - client.1
  seed: 2599
  sha1: f7870454149f22574086dd2ff55d050585136531
  sleep_before_teardown: 0
  subset: 111/120000
  suite: orch
  suite_branch: wip-pdonnell-testing-20260219.182737-tentacle
  suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa
  suite_relpath: qa
  suite_repo: https://github.com/ceph/ceph-ci.git
  suite_sha1: f7870454149f22574086dd2ff55d050585136531
  targets:
    trial160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPl6v1YpJ4UvCi6uV4sAvfjPwuLwQKB9/B5wdFqVDmWQXl3sodZ/d3wHWXSAixga46ld24A6fbQ6Y0QGmst4L+E=
    trial192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBADEQhwziJjcH2Jq+0NcJdD9OUW424724925/fnc8ExqShx11ogH6r+2jyw3YPSeu5TWpalyy/nLhhiI5EHJn6U=
  tasks:
  - pexec:
      all:
      - sudo dnf remove nvme-cli -y
      - sudo dnf install nvmetcli nvme-cli -y
  - nvme_loop: null
  - cephadm:
      roleless: true
  - cephadm.shell:
      host.a:
      - ceph orch status
      - ceph orch ps
      - ceph orch ls
      - ceph orch host ls
      - ceph orch device ls
  - cephadm.shell:
      host.a:
      - ceph orch apply rbd-mirror "--placement=*"
      - ceph orch apply cephfs-mirror "--placement=*"
  - cephadm.wait_for_service:
      service: rbd-mirror
  - cephadm.wait_for_service:
      service: cephfs-mirror
  - cephadm.shell:
      host.a:
      - stat -c '%u %g' /var/log/ceph | grep '167 167'
      - ceph orch status
      - ceph orch ps
      - ceph orch ls
      - ceph orch host ls
      - ceph orch device ls
      - ceph orch ls | grep '^osd.all-available-devices '
  teuthology:
    fragments_dropped: []
    meta: {}
    postmerge: []
  teuthology_branch: main
  teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38
  timestamp: 2026-02-20_16:19:15
  tube: trial
  user: yuriw
  verbose: true
  worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699
2026-02-20T22:35:50.333 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa; will attempt to use it
2026-02-20T22:35:50.333 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f7870454149f22574086dd2ff55d050585136531/qa/tasks
2026-02-20T22:35:50.333 INFO:teuthology.run_tasks:Running task internal.check_packages...
2026-02-20T22:35:50.335 INFO:teuthology.task.internal:Checking packages...
2026-02-20T22:35:50.335 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f7870454149f22574086dd2ff55d050585136531'
2026-02-20T22:35:50.335 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch
2026-02-20T22:35:50.335 INFO:teuthology.packaging:ref: None
2026-02-20T22:35:50.335 INFO:teuthology.packaging:tag: None
2026-02-20T22:35:50.335 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20260219.182737-tentacle
2026-02-20T22:35:50.335 INFO:teuthology.packaging:sha1: f7870454149f22574086dd2ff55d050585136531
2026-02-20T22:35:50.335 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20260219.182737-tentacle
2026-02-20T22:35:50.453 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-677.gf7870454
2026-02-20T22:35:50.454 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep...
2026-02-20T22:35:50.466 INFO:teuthology.task.internal:no buildpackages task found
2026-02-20T22:35:50.466 INFO:teuthology.run_tasks:Running task internal.save_config...
2026-02-20T22:35:50.481 INFO:teuthology.task.internal:Saving configuration
2026-02-20T22:35:50.491 INFO:teuthology.run_tasks:Running task internal.check_lock...
2026-02-20T22:35:50.503 INFO:teuthology.task.internal.check_lock:Checking locks...
2026-02-20T22:35:50.582 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial160.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 22:32:15.429961', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPl6v1YpJ4UvCi6uV4sAvfjPwuLwQKB9/B5wdFqVDmWQXl3sodZ/d3wHWXSAixga46ld24A6fbQ6Y0QGmst4L+E='}
2026-02-20T22:35:50.637 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial192.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 22:32:15.430616', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBADEQhwziJjcH2Jq+0NcJdD9OUW424724925/fnc8ExqShx11ogH6r+2jyw3YPSeu5TWpalyy/nLhhiI5EHJn6U='}
2026-02-20T22:35:50.637 INFO:teuthology.run_tasks:Running task internal.add_remotes...
2026-02-20T22:35:50.649 INFO:teuthology.task.internal:roles: ubuntu@trial160.front.sepia.ceph.com - ['host.a', 'client.0']
2026-02-20T22:35:50.649 INFO:teuthology.task.internal:roles: ubuntu@trial192.front.sepia.ceph.com - ['host.b', 'client.1']
2026-02-20T22:35:50.649 INFO:teuthology.run_tasks:Running task console_log...
2026-02-20T22:35:50.781 DEBUG:teuthology.exit:Installing handler: Handler(exiter=<teuthology.exit.Exiter object at 0x78604376a540>, func=<function ConsoleLog.start_logging.<locals>.kill_console_loggers at 0x786042cff740>, signals=[15])
2026-02-20T22:35:50.781 INFO:teuthology.run_tasks:Running task internal.connect...
2026-02-20T22:35:50.794 INFO:teuthology.task.internal:Opening connections...
2026-02-20T22:35:50.794 DEBUG:teuthology.task.internal:connecting to ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:35:50.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T22:35:50.855 DEBUG:teuthology.task.internal:connecting to ubuntu@trial192.front.sepia.ceph.com
2026-02-20T22:35:50.855 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T22:35:50.917 INFO:teuthology.run_tasks:Running task internal.push_inventory...
2026-02-20T22:35:50.929 DEBUG:teuthology.orchestra.run.trial160:> uname -m
2026-02-20T22:35:50.945 INFO:teuthology.orchestra.run.trial160.stdout:x86_64
2026-02-20T22:35:50.945 DEBUG:teuthology.orchestra.run.trial160:> cat /etc/os-release
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:NAME="CentOS Stream"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:VERSION="9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:ID="centos"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:ID_LIKE="rhel fedora"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:VERSION_ID="9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:PLATFORM_ID="platform:el9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:ANSI_COLOR="0;31"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:LOGO="fedora-logo-icon"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:HOME_URL="https://centos.org/"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T22:35:51.000 INFO:teuthology.orchestra.run.trial160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T22:35:51.001 INFO:teuthology.lock.ops:Updating trial160.front.sepia.ceph.com on lock server
2026-02-20T22:35:51.059 DEBUG:teuthology.orchestra.run.trial192:> uname -m
2026-02-20T22:35:51.074 INFO:teuthology.orchestra.run.trial192.stdout:x86_64
2026-02-20T22:35:51.074 DEBUG:teuthology.orchestra.run.trial192:> cat /etc/os-release
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:NAME="CentOS Stream"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:VERSION="9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:ID="centos"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:ID_LIKE="rhel fedora"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:VERSION_ID="9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:PLATFORM_ID="platform:el9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:PRETTY_NAME="CentOS Stream 9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:ANSI_COLOR="0;31"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:LOGO="fedora-logo-icon"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:CPE_NAME="cpe:/o:centos:centos:9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:HOME_URL="https://centos.org/"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:BUG_REPORT_URL="https://issues.redhat.com/"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9"
2026-02-20T22:35:51.130 INFO:teuthology.orchestra.run.trial192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
2026-02-20T22:35:51.130 INFO:teuthology.lock.ops:Updating trial192.front.sepia.ceph.com on lock server
2026-02-20T22:35:51.202 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles...
2026-02-20T22:35:51.215 INFO:teuthology.run_tasks:Running task internal.check_conflict...
2026-02-20T22:35:51.228 INFO:teuthology.task.internal:Checking for old test directory...
2026-02-20T22:35:51.229 DEBUG:teuthology.orchestra.run.trial160:> test '!' -e /home/ubuntu/cephtest
2026-02-20T22:35:51.230 DEBUG:teuthology.orchestra.run.trial192:> test '!' -e /home/ubuntu/cephtest
2026-02-20T22:35:51.245 INFO:teuthology.run_tasks:Running task internal.check_ceph_data...
2026-02-20T22:35:51.256 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph...
2026-02-20T22:35:51.257 DEBUG:teuthology.orchestra.run.trial160:> test -z $(ls -A /var/lib/ceph)
2026-02-20T22:35:51.286 DEBUG:teuthology.orchestra.run.trial192:> test -z $(ls -A /var/lib/ceph)
2026-02-20T22:35:51.302 INFO:teuthology.run_tasks:Running task internal.vm_setup...
2026-02-20T22:35:51.460 INFO:teuthology.run_tasks:Running task kernel...
2026-02-20T22:35:51.478 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}
2026-02-20T22:35:51.479 INFO:teuthology.task.kernel:config {'host.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'host.b': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300
2026-02-20T22:35:51.479 DEBUG:teuthology.orchestra.run.trial160:> test -f /run/.containerenv -o -f /.dockerenv
2026-02-20T22:35:51.479 DEBUG:teuthology.orchestra.run.trial192:> test -f /run/.containerenv -o -f /.dockerenv
2026-02-20T22:35:51.493 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:35:51.493 DEBUG:teuthology.orchestra.run.trial160:> uname -r
2026-02-20T22:35:51.494 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:35:51.494 DEBUG:teuthology.orchestra.run.trial192:> uname -r
2026-02-20T22:35:51.550 INFO:teuthology.orchestra.run.trial160.stdout:5.14.0-681.el9.x86_64
2026-02-20T22:35:51.550 INFO:teuthology.task.kernel:Running kernel on trial160: 5.14.0-681.el9.x86_64
2026-02-20T22:35:51.550 DEBUG:teuthology.orchestra.run.trial160:> sudo yum install -y kernel
2026-02-20T22:35:51.551 INFO:teuthology.orchestra.run.trial192.stdout:5.14.0-681.el9.x86_64
2026-02-20T22:35:51.551 INFO:teuthology.task.kernel:Running kernel on trial192: 5.14.0-681.el9.x86_64
2026-02-20T22:35:51.551 DEBUG:teuthology.orchestra.run.trial192:> sudo yum install -y kernel
2026-02-20T22:35:53.041 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - BaseOS                        8.0 MB/s | 8.9 MB     00:01
2026-02-20T22:35:53.253 INFO:teuthology.orchestra.run.trial192.stdout:CentOS Stream 9 - BaseOS                        6.8 MB/s | 8.9 MB     00:01
2026-02-20T22:35:56.046 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - AppStream                      11 MB/s |  27 MB     00:02
2026-02-20T22:35:56.398 INFO:teuthology.orchestra.run.trial192.stdout:CentOS Stream 9 - AppStream                      10 MB/s |  27 MB     00:02
2026-02-20T22:35:59.707 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - CRB                           6.1 MB/s | 7.8 MB     00:01
2026-02-20T22:36:00.565 INFO:teuthology.orchestra.run.trial160.stdout:CentOS Stream 9 - Extras packages               154 kB/s |  20 kB     00:00
2026-02-20T22:36:01.452 INFO:teuthology.orchestra.run.trial160.stdout:Extra Packages for Enterprise Linux              25 MB/s |  20 MB     00:00
2026-02-20T22:36:04.664 INFO:teuthology.orchestra.run.trial160.stdout:lab-extras                                      2.5 MB/s |  50 kB     00:00
2026-02-20T22:36:05.669 INFO:teuthology.orchestra.run.trial160.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed.
2026-02-20T22:36:05.669 INFO:teuthology.orchestra.run.trial160.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed.
2026-02-20T22:36:05.688 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved.
2026-02-20T22:36:05.689 INFO:teuthology.orchestra.run.trial160.stdout:Nothing to do.
2026-02-20T22:36:05.689 INFO:teuthology.orchestra.run.trial160.stdout:Complete!
2026-02-20T22:36:05.712 DEBUG:teuthology.orchestra.run.trial160:> echo no | sudo yum reinstall kernel || true
2026-02-20T22:36:06.009 INFO:teuthology.orchestra.run.trial160.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 10:36:04 PM UTC.
2026-02-20T22:36:06.089 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved.
2026-02-20T22:36:06.090 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.090 INFO:teuthology.orchestra.run.trial160.stdout: Package         Architecture    Version                  Repository       Size
2026-02-20T22:36:06.090 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.090 INFO:teuthology.orchestra.run.trial160.stdout:Reinstalling:
2026-02-20T22:36:06.090 INFO:teuthology.orchestra.run.trial160.stdout: kernel          x86_64          5.14.0-681.el9           baseos          803 k
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:Transaction Summary
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:Total download size: 803 k
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stdout:Installed size: 0
2026-02-20T22:36:06.091 INFO:teuthology.orchestra.run.trial160.stderr:Operation aborted.
2026-02-20T22:36:06.110 INFO:teuthology.orchestra.run.trial160.stdout:Is this ok [y/N]:
2026-02-20T22:36:06.110 DEBUG:teuthology.orchestra.run.trial160:> sudo yum reinstall -y kernel || true
2026-02-20T22:36:06.408 INFO:teuthology.orchestra.run.trial160.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 10:36:04 PM UTC.
2026-02-20T22:36:06.488 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved.
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout: Package         Architecture    Version                  Repository       Size
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout:Reinstalling:
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout: kernel          x86_64          5.14.0-681.el9           baseos          803 k
2026-02-20T22:36:06.489 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:Transaction Summary
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:Total download size: 803 k
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:Installed size: 0
2026-02-20T22:36:06.490 INFO:teuthology.orchestra.run.trial160.stdout:Downloading Packages:
2026-02-20T22:36:07.397 INFO:teuthology.orchestra.run.trial192.stdout:CentOS Stream 9 - CRB                           919 kB/s | 7.8 MB     00:08
2026-02-20T22:36:08.001 INFO:teuthology.orchestra.run.trial160.stdout:kernel-5.14.0-681.el9.x86_64.rpm                553 kB/s | 803 kB     00:01
2026-02-20T22:36:08.001 INFO:teuthology.orchestra.run.trial160.stdout:--------------------------------------------------------------------------------
2026-02-20T22:36:08.001 INFO:teuthology.orchestra.run.trial160.stdout:Total                                           531 kB/s | 803 kB     00:01
2026-02-20T22:36:08.011 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction check
2026-02-20T22:36:08.044 INFO:teuthology.orchestra.run.trial160.stdout:Transaction check succeeded.
2026-02-20T22:36:08.045 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction test
2026-02-20T22:36:08.049 INFO:teuthology.orchestra.run.trial160.stdout:Transaction test succeeded.
2026-02-20T22:36:08.049 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction
2026-02-20T22:36:08.073 INFO:teuthology.orchestra.run.trial160.stdout:  Preparing        :                                                        1/1
2026-02-20T22:36:08.077 INFO:teuthology.orchestra.run.trial160.stdout:  Reinstalling     : kernel-5.14.0-681.el9.x86_64                           1/2
2026-02-20T22:36:08.077 INFO:teuthology.orchestra.run.trial160.stdout:  Cleanup          : kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:08.182 INFO:teuthology.orchestra.run.trial160.stdout:  Running scriptlet: kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:08.183 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : kernel-5.14.0-681.el9.x86_64                           1/2
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:Reinstalled:
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:  kernel-5.14.0-681.el9.x86_64
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:36:08.237 INFO:teuthology.orchestra.run.trial160.stdout:Complete!
2026-02-20T22:36:08.265 DEBUG:teuthology.orchestra.run.trial160:> rpm -q kernel | sort -rV | head -n 1
2026-02-20T22:36:08.288 INFO:teuthology.orchestra.run.trial160.stdout:kernel-5.14.0-681.el9.x86_64
2026-02-20T22:36:08.289 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64
2026-02-20T22:36:08.290 INFO:teuthology.task.kernel:Newest distro kernel installed and running
2026-02-20T22:36:08.290 INFO:teuthology.task.kernel:Enabling kdb on host.a...
2026-02-20T22:36:08.290 DEBUG:teuthology.orchestra.run.trial160:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc
2026-02-20T22:36:08.327 INFO:teuthology.orchestra.run.trial192.stdout:CentOS Stream 9 - Extras packages                98 kB/s |  20 kB     00:00
2026-02-20T22:36:08.353 INFO:teuthology.orchestra.run.trial160.stdout:ttyS1
2026-02-20T22:36:08.383 DEBUG:teuthology.parallel:result is None
2026-02-20T22:36:08.560 INFO:teuthology.orchestra.run.trial192.stdout:Extra Packages for Enterprise Linux             119 MB/s |  20 MB     00:00
2026-02-20T22:36:11.752 INFO:teuthology.orchestra.run.trial192.stdout:lab-extras                                      2.5 MB/s |  50 kB     00:00
2026-02-20T22:36:12.768 INFO:teuthology.orchestra.run.trial192.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed.
2026-02-20T22:36:12.768 INFO:teuthology.orchestra.run.trial192.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed.
2026-02-20T22:36:12.787 INFO:teuthology.orchestra.run.trial192.stdout:Dependencies resolved.
2026-02-20T22:36:12.788 INFO:teuthology.orchestra.run.trial192.stdout:Nothing to do.
2026-02-20T22:36:12.788 INFO:teuthology.orchestra.run.trial192.stdout:Complete!
2026-02-20T22:36:12.810 DEBUG:teuthology.orchestra.run.trial192:> echo no | sudo yum reinstall kernel || true
2026-02-20T22:36:13.108 INFO:teuthology.orchestra.run.trial192.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 10:36:11 PM UTC.
2026-02-20T22:36:13.188 INFO:teuthology.orchestra.run.trial192.stdout:Dependencies resolved.
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout: Package         Architecture    Version                  Repository       Size
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout:Reinstalling:
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout: kernel          x86_64          5.14.0-681.el9           baseos          803 k
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:13.189 INFO:teuthology.orchestra.run.trial192.stdout:Transaction Summary
2026-02-20T22:36:13.190 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.190 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:13.190 INFO:teuthology.orchestra.run.trial192.stdout:Total download size: 803 k
2026-02-20T22:36:13.190 INFO:teuthology.orchestra.run.trial192.stdout:Installed size: 0
2026-02-20T22:36:13.190 INFO:teuthology.orchestra.run.trial192.stderr:Operation aborted.
2026-02-20T22:36:13.209 INFO:teuthology.orchestra.run.trial192.stdout:Is this ok [y/N]:
2026-02-20T22:36:13.210 DEBUG:teuthology.orchestra.run.trial192:> sudo yum reinstall -y kernel || true
2026-02-20T22:36:13.511 INFO:teuthology.orchestra.run.trial192.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 10:36:11 PM UTC.
2026-02-20T22:36:13.591 INFO:teuthology.orchestra.run.trial192.stdout:Dependencies resolved.
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout: Package         Architecture    Version                  Repository       Size
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:Reinstalling:
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout: kernel          x86_64          5.14.0-681.el9           baseos          803 k
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:Transaction Summary
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:13.592 INFO:teuthology.orchestra.run.trial192.stdout:Total download size: 803 k
2026-02-20T22:36:13.593 INFO:teuthology.orchestra.run.trial192.stdout:Installed size: 0
2026-02-20T22:36:13.593 INFO:teuthology.orchestra.run.trial192.stdout:Downloading Packages:
2026-02-20T22:36:14.141 INFO:teuthology.orchestra.run.trial192.stdout:kernel-5.14.0-681.el9.x86_64.rpm                3.2 MB/s | 803 kB     00:00
2026-02-20T22:36:14.141 INFO:teuthology.orchestra.run.trial192.stdout:--------------------------------------------------------------------------------
2026-02-20T22:36:14.141 INFO:teuthology.orchestra.run.trial192.stdout:Total                                           1.4 MB/s | 803 kB     00:00
2026-02-20T22:36:14.151 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction check
2026-02-20T22:36:14.185 INFO:teuthology.orchestra.run.trial192.stdout:Transaction check succeeded.
2026-02-20T22:36:14.185 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction test
2026-02-20T22:36:14.190 INFO:teuthology.orchestra.run.trial192.stdout:Transaction test succeeded.
2026-02-20T22:36:14.190 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction
2026-02-20T22:36:14.213 INFO:teuthology.orchestra.run.trial192.stdout:  Preparing        :                                                        1/1
2026-02-20T22:36:14.217 INFO:teuthology.orchestra.run.trial192.stdout:  Reinstalling     : kernel-5.14.0-681.el9.x86_64                           1/2
2026-02-20T22:36:14.217 INFO:teuthology.orchestra.run.trial192.stdout:  Cleanup          : kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:14.326 INFO:teuthology.orchestra.run.trial192.stdout:  Running scriptlet: kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:14.326 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : kernel-5.14.0-681.el9.x86_64                           1/2
2026-02-20T22:36:14.379 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : kernel-5.14.0-681.el9.x86_64                           2/2
2026-02-20T22:36:14.379 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:14.380 INFO:teuthology.orchestra.run.trial192.stdout:Reinstalled:
2026-02-20T22:36:14.380 INFO:teuthology.orchestra.run.trial192.stdout:  kernel-5.14.0-681.el9.x86_64
2026-02-20T22:36:14.380 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:36:14.380 INFO:teuthology.orchestra.run.trial192.stdout:Complete!
2026-02-20T22:36:14.406 DEBUG:teuthology.orchestra.run.trial192:> rpm -q kernel | sort -rV | head -n 1
2026-02-20T22:36:14.471 INFO:teuthology.orchestra.run.trial192.stdout:kernel-5.14.0-681.el9.x86_64
2026-02-20T22:36:14.472 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64
2026-02-20T22:36:14.472 INFO:teuthology.task.kernel:Newest distro kernel installed and running
2026-02-20T22:36:14.472 INFO:teuthology.task.kernel:Enabling kdb on host.b...
2026-02-20T22:36:14.472 DEBUG:teuthology.orchestra.run.trial192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc
2026-02-20T22:36:14.537 INFO:teuthology.orchestra.run.trial192.stdout:ttyS1
2026-02-20T22:36:14.560 DEBUG:teuthology.parallel:result is None
2026-02-20T22:36:14.560 INFO:teuthology.run_tasks:Running task internal.base...
2026-02-20T22:36:14.571 INFO:teuthology.task.internal:Creating test directory...
2026-02-20T22:36:14.571 DEBUG:teuthology.orchestra.run.trial160:> mkdir -p -m0755 -- /home/ubuntu/cephtest
2026-02-20T22:36:14.573 DEBUG:teuthology.orchestra.run.trial192:> mkdir -p -m0755 -- /home/ubuntu/cephtest
2026-02-20T22:36:14.615 INFO:teuthology.run_tasks:Running task internal.archive_upload...
2026-02-20T22:36:14.662 INFO:teuthology.run_tasks:Running task internal.archive...
2026-02-20T22:36:14.692 INFO:teuthology.task.internal:Creating archive directory...
2026-02-20T22:36:14.693 DEBUG:teuthology.orchestra.run.trial160:> install -d -m0755 -- /home/ubuntu/cephtest/archive
2026-02-20T22:36:14.711 DEBUG:teuthology.orchestra.run.trial192:> install -d -m0755 -- /home/ubuntu/cephtest/archive
2026-02-20T22:36:14.731 INFO:teuthology.run_tasks:Running task internal.coredump...
2026-02-20T22:36:14.748 INFO:teuthology.task.internal:Enabling coredump saving...
2026-02-20T22:36:14.748 DEBUG:teuthology.orchestra.run.trial160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf
2026-02-20T22:36:14.775 DEBUG:teuthology.orchestra.run.trial192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf
2026-02-20T22:36:14.798 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core
2026-02-20T22:36:14.801 INFO:teuthology.orchestra.run.trial192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core
2026-02-20T22:36:14.807 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core
2026-02-20T22:36:14.810 INFO:teuthology.orchestra.run.trial192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core
2026-02-20T22:36:14.812 INFO:teuthology.run_tasks:Running task internal.sudo...
2026-02-20T22:36:14.825 INFO:teuthology.task.internal:Configuring sudo...
2026-02-20T22:36:14.826 DEBUG:teuthology.orchestra.run.trial160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers
2026-02-20T22:36:14.851 DEBUG:teuthology.orchestra.run.trial192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers
2026-02-20T22:36:14.877 INFO:teuthology.run_tasks:Running task internal.syslog...
2026-02-20T22:36:14.893 INFO:teuthology.task.internal.syslog:Starting syslog monitoring...
2026-02-20T22:36:14.894 DEBUG:teuthology.orchestra.run.trial160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog
2026-02-20T22:36:14.916 DEBUG:teuthology.orchestra.run.trial192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog
2026-02-20T22:36:14.933 DEBUG:teuthology.orchestra.run.trial160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:36:15.090 DEBUG:teuthology.orchestra.run.trial160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:36:15.116 DEBUG:teuthology.orchestra.run.trial160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:36:15.263 DEBUG:teuthology.orchestra.run.trial160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:36:15.287 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:36:15.287 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf
2026-02-20T22:36:15.351 DEBUG:teuthology.orchestra.run.trial192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:36:15.469 DEBUG:teuthology.orchestra.run.trial192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:36:15.494 DEBUG:teuthology.orchestra.run.trial192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:36:15.647 DEBUG:teuthology.orchestra.run.trial192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:36:15.672 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:36:15.672 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf
2026-02-20T22:36:15.738 DEBUG:teuthology.orchestra.run.trial160:> sudo service rsyslog restart
2026-02-20T22:36:15.740 DEBUG:teuthology.orchestra.run.trial192:> sudo service rsyslog restart
2026-02-20T22:36:15.767 INFO:teuthology.orchestra.run.trial160.stderr:Redirecting to /bin/systemctl restart rsyslog.service
2026-02-20T22:36:15.805 INFO:teuthology.orchestra.run.trial192.stderr:Redirecting to /bin/systemctl restart rsyslog.service
2026-02-20T22:36:16.151 INFO:teuthology.run_tasks:Running task internal.timer...
2026-02-20T22:36:16.164 INFO:teuthology.task.internal:Starting timer...
2026-02-20T22:36:16.164 INFO:teuthology.run_tasks:Running task pcp...
2026-02-20T22:36:16.186 INFO:teuthology.run_tasks:Running task selinux...
2026-02-20T22:36:16.207 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']}
2026-02-20T22:36:16.207 DEBUG:teuthology.orchestra.run.trial160:> sudo service auditd rotate
2026-02-20T22:36:16.237 INFO:teuthology.orchestra.run.trial160.stdout:Rotating logs:
2026-02-20T22:36:16.239 DEBUG:teuthology.orchestra.run.trial192:> sudo service auditd rotate
2026-02-20T22:36:16.268 INFO:teuthology.orchestra.run.trial192.stdout:Rotating logs:
2026-02-20T22:36:16.270 DEBUG:teuthology.task.selinux:Getting current SELinux state
2026-02-20T22:36:16.270 DEBUG:teuthology.orchestra.run.trial160:> /usr/sbin/getenforce
2026-02-20T22:36:16.296 INFO:teuthology.orchestra.run.trial160.stdout:Permissive
2026-02-20T22:36:16.296 DEBUG:teuthology.orchestra.run.trial192:> /usr/sbin/getenforce
2026-02-20T22:36:16.325 INFO:teuthology.orchestra.run.trial192.stdout:Permissive
2026-02-20T22:36:16.325 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial160.front.sepia.ceph.com': 'permissive', 'ubuntu@trial192.front.sepia.ceph.com': 'permissive'}
2026-02-20T22:36:16.325 DEBUG:teuthology.orchestra.run.trial160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0
2026-02-20T22:36:16.360 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:36:16.361 DEBUG:teuthology.orchestra.run.trial192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0
2026-02-20T22:36:16.390 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:36:16.391 INFO:teuthology.task.selinux:Putting SELinux into permissive mode
2026-02-20T22:36:16.391 DEBUG:teuthology.orchestra.run.trial160:> sudo /usr/sbin/setenforce permissive
2026-02-20T22:36:16.424 DEBUG:teuthology.orchestra.run.trial192:> sudo /usr/sbin/setenforce permissive
2026-02-20T22:36:16.454 INFO:teuthology.run_tasks:Running task ansible.cephlab...
2026-02-20T22:36:16.469 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main
2026-02-20T22:36:16.481 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}]
2026-02-20T22:36:16.482 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial160.front.sepia.ceph.com,trial192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml
2026-02-20T22:39:02.925 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial160.front.sepia.ceph.com'), Remote(name='ubuntu@trial192.front.sepia.ceph.com')]
2026-02-20T22:39:02.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial160.front.sepia.ceph.com'
2026-02-20T22:39:02.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T22:39:03.036 DEBUG:teuthology.orchestra.run.trial160:> true
2026-02-20T22:39:03.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial160.front.sepia.ceph.com'
2026-02-20T22:39:03.111 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial192.front.sepia.ceph.com'
2026-02-20T22:39:03.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60}
2026-02-20T22:39:03.207 DEBUG:teuthology.orchestra.run.trial192:> true
2026-02-20T22:39:03.288 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial192.front.sepia.ceph.com'
2026-02-20T22:39:03.288 INFO:teuthology.run_tasks:Running task clock...
2026-02-20T22:39:03.302 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew...
2026-02-20T22:39:03.302 INFO:teuthology.orchestra.run:Running command with timeout 360
2026-02-20T22:39:03.303 DEBUG:teuthology.orchestra.run.trial160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true
2026-02-20T22:39:03.305 INFO:teuthology.orchestra.run:Running command with timeout 360
2026-02-20T22:39:03.305 DEBUG:teuthology.orchestra.run.trial192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true
2026-02-20T22:39:03.331 INFO:teuthology.orchestra.run.trial160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded.
2026-02-20T22:39:03.344 INFO:teuthology.orchestra.run.trial160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded.
2026-02-20T22:39:03.356 INFO:teuthology.orchestra.run.trial192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded.
2026-02-20T22:39:03.365 INFO:teuthology.orchestra.run.trial160.stderr:sudo: ntpd: command not found
2026-02-20T22:39:03.368 INFO:teuthology.orchestra.run.trial192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded.
2026-02-20T22:39:03.374 INFO:teuthology.orchestra.run.trial160.stdout:506 Cannot talk to daemon
2026-02-20T22:39:03.386 INFO:teuthology.orchestra.run.trial160.stderr:Failed to start ntp.service: Unit ntp.service not found.
2026-02-20T22:39:03.389 INFO:teuthology.orchestra.run.trial192.stderr:sudo: ntpd: command not found
2026-02-20T22:39:03.397 INFO:teuthology.orchestra.run.trial160.stderr:Failed to start ntpd.service: Unit ntpd.service not found.
2026-02-20T22:39:03.399 INFO:teuthology.orchestra.run.trial192.stdout:506 Cannot talk to daemon
2026-02-20T22:39:03.411 INFO:teuthology.orchestra.run.trial192.stderr:Failed to start ntp.service: Unit ntp.service not found.
2026-02-20T22:39:03.424 INFO:teuthology.orchestra.run.trial192.stderr:Failed to start ntpd.service: Unit ntpd.service not found.
2026-02-20T22:39:03.443 INFO:teuthology.orchestra.run.trial160.stderr:bash: line 1: ntpq: command not found
2026-02-20T22:39:03.444 INFO:teuthology.orchestra.run.trial160.stdout:MS Name/IP address         Stratum Poll Reach LastRx Last sample
2026-02-20T22:39:03.444 INFO:teuthology.orchestra.run.trial160.stdout:===============================================================================
2026-02-20T22:39:03.470 INFO:teuthology.orchestra.run.trial192.stderr:bash: line 1: ntpq: command not found
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:MS Name/IP address         Stratum Poll Reach LastRx Last sample
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:===============================================================================
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:^? 64.44.115.65                  0   6     0     -     +0ns[   +0ns] +/-    0ns
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:^? 163-123-153-14.ip4.evals>     0   6     0     -     +0ns[   +0ns] +/-    0ns
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:^? 44.190.5.123                  0   6     0     -     +0ns[   +0ns] +/-    0ns
2026-02-20T22:39:03.490 INFO:teuthology.orchestra.run.trial192.stdout:^? time.cloudflare.com           0   6     0     -     +0ns[   +0ns] +/-    0ns
2026-02-20T22:39:03.491 INFO:teuthology.run_tasks:Running task pexec...
2026-02-20T22:39:03.505 INFO:teuthology.task.pexec:Executing custom commands...
2026-02-20T22:39:03.506 DEBUG:teuthology.orchestra.run.trial160:> TESTDIR=/home/ubuntu/cephtest bash -s
2026-02-20T22:39:03.506 DEBUG:teuthology.orchestra.run.trial192:> TESTDIR=/home/ubuntu/cephtest bash -s
2026-02-20T22:39:03.508 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:39:03.508 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y
2026-02-20T22:39:03.508 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y
2026-02-20T22:39:03.533 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial192.front.sepia.ceph.com
2026-02-20T22:39:03.534 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y
2026-02-20T22:39:03.534 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y
2026-02-20T22:39:03.677 INFO:teuthology.orchestra.run.trial160.stdout:No match for argument: nvme-cli
2026-02-20T22:39:03.678 INFO:teuthology.orchestra.run.trial160.stderr:No packages marked for removal.
2026-02-20T22:39:03.679 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved.
2026-02-20T22:39:03.680 INFO:teuthology.orchestra.run.trial160.stdout:Nothing to do.
2026-02-20T22:39:03.680 INFO:teuthology.orchestra.run.trial160.stdout:Complete!
2026-02-20T22:39:03.705 INFO:teuthology.orchestra.run.trial192.stdout:No match for argument: nvme-cli
2026-02-20T22:39:03.705 INFO:teuthology.orchestra.run.trial192.stderr:No packages marked for removal.
2026-02-20T22:39:03.707 INFO:teuthology.orchestra.run.trial192.stdout:Dependencies resolved.
2026-02-20T22:39:03.707 INFO:teuthology.orchestra.run.trial192.stdout:Nothing to do.
2026-02-20T22:39:03.707 INFO:teuthology.orchestra.run.trial192.stdout:Complete!
2026-02-20T22:39:03.985 INFO:teuthology.orchestra.run.trial160.stdout:Last metadata expiration check: 0:00:29 ago on Fri 20 Feb 2026 10:38:34 PM UTC.
2026-02-20T22:39:04.006 INFO:teuthology.orchestra.run.trial192.stdout:Last metadata expiration check: 0:00:28 ago on Fri 20 Feb 2026 10:38:35 PM UTC.
2026-02-20T22:39:04.066 INFO:teuthology.orchestra.run.trial160.stdout:Dependencies resolved.
2026-02-20T22:39:04.067 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:39:04.067 INFO:teuthology.orchestra.run.trial160.stdout: Package                   Architecture Version              Repository    Size
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout:Installing:
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: nvme-cli                  x86_64       2.16-1.el9           baseos       1.2 M
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: nvmetcli                  noarch       0.8-3.el9            baseos        44 k
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout:Installing dependencies:
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: python3-configshell       noarch       1:1.1.30-1.el9       baseos        72 k
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: python3-kmod              x86_64       0.9-32.el9           baseos        84 k
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: python3-pyparsing         noarch       2.4.7-9.el9          baseos       150 k
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout: python3-urwid             x86_64       2.1.2-4.el9          baseos       837 k
2026-02-20T22:39:04.068 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:Transaction Summary
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:================================================================================
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:Install  6 Packages
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:Total download size: 2.3 M
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:Installed size: 11 M
2026-02-20T22:39:04.069 INFO:teuthology.orchestra.run.trial160.stdout:Downloading Packages:
2026-02-20T22:39:04.088 INFO:teuthology.orchestra.run.trial192.stdout:Dependencies resolved.
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: Package                   Architecture Version              Repository    Size
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout:Installing:
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: nvme-cli                  x86_64       2.16-1.el9           baseos       1.2 M
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: nvmetcli                  noarch       0.8-3.el9            baseos        44 k
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout:Installing dependencies:
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: python3-configshell       noarch       1:1.1.30-1.el9       baseos        72 k
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: python3-kmod              x86_64       0.9-32.el9           baseos        84 k
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: python3-pyparsing         noarch       2.4.7-9.el9          baseos       150 k
2026-02-20T22:39:04.089 INFO:teuthology.orchestra.run.trial192.stdout: python3-urwid             x86_64       2.1.2-4.el9          baseos       837 k
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:Transaction Summary
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:================================================================================
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:Install  6 Packages
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:Total download size: 2.3 M
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:Installed size: 11 M
2026-02-20T22:39:04.090 INFO:teuthology.orchestra.run.trial192.stdout:Downloading Packages:
2026-02-20T22:39:04.229 INFO:teuthology.orchestra.run.trial160.stdout:(1/6): nvmetcli-0.8-3.el9.noarch.rpm            445 kB/s |  44 kB     00:00
2026-02-20T22:39:04.243 INFO:teuthology.orchestra.run.trial160.stdout:(2/6): python3-configshell-1.1.30-1.el9.noarch. 640 kB/s |  72 kB     00:00
2026-02-20T22:39:04.284 INFO:teuthology.orchestra.run.trial160.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm       1.5 MB/s |  84 kB     00:00
2026-02-20T22:39:04.309 INFO:teuthology.orchestra.run.trial160.stdout:(4/6): python3-pyparsing-2.4.7-9.el9.noarch.rpm 2.2 MB/s | 150 kB     00:00
2026-02-20T22:39:04.341 INFO:teuthology.orchestra.run.trial160.stdout:(5/6): nvme-cli-2.16-1.el9.x86_64.rpm           5.5 MB/s | 1.2 MB     00:00
2026-02-20T22:39:04.387 INFO:teuthology.orchestra.run.trial160.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm     8.0 MB/s | 837 kB     00:00
2026-02-20T22:39:04.387 INFO:teuthology.orchestra.run.trial160.stdout:--------------------------------------------------------------------------------
2026-02-20T22:39:04.387 INFO:teuthology.orchestra.run.trial160.stdout:Total                                           7.2 MB/s | 2.3 MB     00:00
2026-02-20T22:39:04.438 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction check
2026-02-20T22:39:04.446 INFO:teuthology.orchestra.run.trial160.stdout:Transaction check succeeded.
2026-02-20T22:39:04.446 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction test
2026-02-20T22:39:04.460 INFO:teuthology.orchestra.run.trial192.stdout:(1/6): python3-configshell-1.1.30-1.el9.noarch. 1.0 MB/s |  72 kB     00:00
2026-02-20T22:39:04.462 INFO:teuthology.orchestra.run.trial192.stdout:(2/6): nvmetcli-0.8-3.el9.noarch.rpm            628 kB/s |  44 kB     00:00
2026-02-20T22:39:04.504 INFO:teuthology.orchestra.run.trial192.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm       1.9 MB/s |  84 kB     00:00
2026-02-20T22:39:04.506 INFO:teuthology.orchestra.run.trial192.stdout:(4/6): python3-pyparsing-2.4.7-9.el9.noarch.rpm 3.3 MB/s | 150 kB     00:00
2026-02-20T22:39:04.510 INFO:teuthology.orchestra.run.trial160.stdout:Transaction test succeeded.
2026-02-20T22:39:04.510 INFO:teuthology.orchestra.run.trial160.stdout:Running transaction
2026-02-20T22:39:04.544 INFO:teuthology.orchestra.run.trial192.stdout:(5/6): nvme-cli-2.16-1.el9.x86_64.rpm           7.6 MB/s | 1.2 MB     00:00
2026-02-20T22:39:04.572 INFO:teuthology.orchestra.run.trial192.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm      12 MB/s | 837 kB     00:00
2026-02-20T22:39:04.572 INFO:teuthology.orchestra.run.trial192.stdout:--------------------------------------------------------------------------------
2026-02-20T22:39:04.572 INFO:teuthology.orchestra.run.trial192.stdout:Total                                           4.8 MB/s | 2.3 MB     00:00
2026-02-20T22:39:04.629 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction check
2026-02-20T22:39:04.637 INFO:teuthology.orchestra.run.trial192.stdout:Transaction check succeeded.
2026-02-20T22:39:04.637 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction test
2026-02-20T22:39:04.642 INFO:teuthology.orchestra.run.trial160.stdout:  Preparing        :                                                        1/1
2026-02-20T22:39:04.650 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : python3-urwid-2.1.2-4.el9.x86_64                       1/6
2026-02-20T22:39:04.659 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : python3-pyparsing-2.4.7-9.el9.noarch                   2/6
2026-02-20T22:39:04.664 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : python3-configshell-1:1.1.30-1.el9.noarch              3/6
2026-02-20T22:39:04.670 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : python3-kmod-0.9-32.el9.x86_64                         4/6
2026-02-20T22:39:04.671 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : nvmetcli-0.8-3.el9.noarch                              5/6
2026-02-20T22:39:04.703 INFO:teuthology.orchestra.run.trial192.stdout:Transaction test succeeded.
2026-02-20T22:39:04.703 INFO:teuthology.orchestra.run.trial192.stdout:Running transaction
2026-02-20T22:39:04.806 INFO:teuthology.orchestra.run.trial160.stdout:  Running scriptlet: nvmetcli-0.8-3.el9.noarch                              5/6
2026-02-20T22:39:04.808 INFO:teuthology.orchestra.run.trial160.stdout:  Installing       : nvme-cli-2.16-1.el9.x86_64                             6/6
2026-02-20T22:39:04.835 INFO:teuthology.orchestra.run.trial192.stdout:  Preparing        :                                                        1/1
2026-02-20T22:39:04.843 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : python3-urwid-2.1.2-4.el9.x86_64                       1/6
2026-02-20T22:39:04.852 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : python3-pyparsing-2.4.7-9.el9.noarch                   2/6
2026-02-20T22:39:04.857 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : python3-configshell-1:1.1.30-1.el9.noarch              3/6
2026-02-20T22:39:04.863 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : python3-kmod-0.9-32.el9.x86_64                         4/6
2026-02-20T22:39:04.864 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : nvmetcli-0.8-3.el9.noarch                              5/6
2026-02-20T22:39:04.995 INFO:teuthology.orchestra.run.trial192.stdout:  Running scriptlet: nvmetcli-0.8-3.el9.noarch                              5/6
2026-02-20T22:39:04.998 INFO:teuthology.orchestra.run.trial192.stdout:  Installing       : nvme-cli-2.16-1.el9.x86_64                             6/6
2026-02-20T22:39:05.091 INFO:teuthology.orchestra.run.trial160.stdout:  Running scriptlet: nvme-cli-2.16-1.el9.x86_64                             6/6
2026-02-20T22:39:05.091 INFO:teuthology.orchestra.run.trial160.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service.
2026-02-20T22:39:05.091 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:05.300 INFO:teuthology.orchestra.run.trial192.stdout:  Running scriptlet: nvme-cli-2.16-1.el9.x86_64                             6/6
2026-02-20T22:39:05.300 INFO:teuthology.orchestra.run.trial192.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service.
2026-02-20T22:39:05.300 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:05.323 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : nvme-cli-2.16-1.el9.x86_64                             1/6
2026-02-20T22:39:05.323 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : nvmetcli-0.8-3.el9.noarch                              2/6
2026-02-20T22:39:05.323 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : python3-configshell-1:1.1.30-1.el9.noarch              3/6
2026-02-20T22:39:05.323 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : python3-kmod-0.9-32.el9.x86_64                         4/6
2026-02-20T22:39:05.323 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : python3-pyparsing-2.4.7-9.el9.noarch                   5/6
2026-02-20T22:39:05.375 INFO:teuthology.orchestra.run.trial160.stdout:  Verifying        : python3-urwid-2.1.2-4.el9.x86_64                       6/6
2026-02-20T22:39:05.375 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:05.375 INFO:teuthology.orchestra.run.trial160.stdout:Installed:
2026-02-20T22:39:05.375 INFO:teuthology.orchestra.run.trial160.stdout:  nvme-cli-2.16-1.el9.x86_64                  nvmetcli-0.8-3.el9.noarch
2026-02-20T22:39:05.375 INFO:teuthology.orchestra.run.trial160.stdout:  python3-configshell-1:1.1.30-1.el9.noarch   python3-kmod-0.9-32.el9.x86_64
2026-02-20T22:39:05.376 INFO:teuthology.orchestra.run.trial160.stdout:  python3-pyparsing-2.4.7-9.el9.noarch        python3-urwid-2.1.2-4.el9.x86_64
2026-02-20T22:39:05.376 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:05.376 INFO:teuthology.orchestra.run.trial160.stdout:Complete!
2026-02-20T22:39:05.406 DEBUG:teuthology.parallel:result is None
2026-02-20T22:39:05.541 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : nvme-cli-2.16-1.el9.x86_64                             1/6
2026-02-20T22:39:05.541 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : nvmetcli-0.8-3.el9.noarch                              2/6
2026-02-20T22:39:05.541 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : python3-configshell-1:1.1.30-1.el9.noarch              3/6
2026-02-20T22:39:05.541 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : python3-kmod-0.9-32.el9.x86_64                         4/6
2026-02-20T22:39:05.541 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : python3-pyparsing-2.4.7-9.el9.noarch                   5/6
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:  Verifying        : python3-urwid-2.1.2-4.el9.x86_64                       6/6
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:Installed:
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:  nvme-cli-2.16-1.el9.x86_64                  nvmetcli-0.8-3.el9.noarch
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:  python3-configshell-1:1.1.30-1.el9.noarch   python3-kmod-0.9-32.el9.x86_64
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:  python3-pyparsing-2.4.7-9.el9.noarch        python3-urwid-2.1.2-4.el9.x86_64
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:05.595 INFO:teuthology.orchestra.run.trial192.stdout:Complete!
2026-02-20T22:39:05.626 DEBUG:teuthology.parallel:result is None
2026-02-20T22:39:05.626 INFO:teuthology.run_tasks:Running task nvme_loop...
2026-02-20T22:39:05.642 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices...
2026-02-20T22:39:05.642 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:05.642 DEBUG:teuthology.orchestra.run.trial160:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:39:05.659 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4']
2026-02-20T22:39:05.660 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_1
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/vg_nvme/lv_1 -> ../dm-0
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1306        Links: 1
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:05.237823499 +0000
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:05.114828015 +0000
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:05.114828015 +0000
2026-02-20T22:39:05.717 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:05.114828015 +0000
2026-02-20T22:39:05.717 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1
2026-02-20T22:39:05.782 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:39:05.782 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:39:05.782 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000187849 s, 2.7 MB/s
2026-02-20T22:39:05.783 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1
2026-02-20T22:39:05.840 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_2
2026-02-20T22:39:05.896 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/vg_nvme/lv_2 -> ../dm-1
2026-02-20T22:39:05.896 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1301        Links: 1
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:05.237823499 +0000
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:05.897 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:05.898 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1
2026-02-20T22:39:05.961 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:39:05.961 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:39:05.961 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000148195 s, 3.5 MB/s
2026-02-20T22:39:05.963 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2
2026-02-20T22:39:06.021 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_3
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/vg_nvme/lv_3 -> ../dm-2
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1289        Links: 1
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:05.238823463 +0000
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.078 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.079 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1
2026-02-20T22:39:06.141 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:39:06.141 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:39:06.141 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000155117 s, 3.3 MB/s
2026-02-20T22:39:06.142 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3
2026-02-20T22:39:06.199 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/vg_nvme/lv_4
2026-02-20T22:39:06.254 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/vg_nvme/lv_4 -> ../dm-3
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1304        Links: 1
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:05.238823463 +0000
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.255 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:05.112828087 +0000
2026-02-20T22:39:06.255 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1
2026-02-20T22:39:06.318 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:39:06.318 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:39:06.318 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000156321 s, 3.3 MB/s
2026-02-20T22:39:06.319 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4
2026-02-20T22:39:06.378 DEBUG:teuthology.orchestra.run.trial160:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype
2026-02-20T22:39:06.485 INFO:teuthology.orchestra.run.trial160.stdout:loop
2026-02-20T22:39:06.486 INFO:tasks.nvme_loop:Connecting nvme_loop trial160:/dev/vg_nvme/lv_1...
2026-02-20T22:39:06.487 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn
2026-02-20T22:39:06.519 INFO:teuthology.orchestra.run.trial160.stdout:1
2026-02-20T22:39:06.545 INFO:teuthology.orchestra.run.trial160.stdout:/dev/vg_nvme/lv_11
2026-02-20T22:39:06.575 INFO:teuthology.orchestra.run.trial160.stdout:connecting to device: nvme3
2026-02-20T22:39:06.581 INFO:tasks.nvme_loop:Connecting nvme_loop trial160:/dev/vg_nvme/lv_2...
2026-02-20T22:39:06.581 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn
2026-02-20T22:39:06.653 INFO:teuthology.orchestra.run.trial160.stdout:1
2026-02-20T22:39:06.680 INFO:teuthology.orchestra.run.trial160.stdout:/dev/vg_nvme/lv_21
2026-02-20T22:39:06.714 INFO:teuthology.orchestra.run.trial160.stdout:connecting to device: nvme4
2026-02-20T22:39:06.716 INFO:tasks.nvme_loop:Connecting nvme_loop trial160:/dev/vg_nvme/lv_3...
2026-02-20T22:39:06.717 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn
2026-02-20T22:39:06.787 INFO:teuthology.orchestra.run.trial160.stdout:1
2026-02-20T22:39:06.814 INFO:teuthology.orchestra.run.trial160.stdout:/dev/vg_nvme/lv_31
2026-02-20T22:39:06.843 INFO:teuthology.orchestra.run.trial160.stdout:connecting to device: nvme5
2026-02-20T22:39:06.845 INFO:tasks.nvme_loop:Connecting nvme_loop trial160:/dev/vg_nvme/lv_4...
2026-02-20T22:39:06.845 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn
2026-02-20T22:39:06.917 INFO:teuthology.orchestra.run.trial160.stdout:1
2026-02-20T22:39:06.941 INFO:teuthology.orchestra.run.trial160.stdout:/dev/vg_nvme/lv_41
2026-02-20T22:39:06.976 INFO:teuthology.orchestra.run.trial160.stdout:connecting to device: nvme6
2026-02-20T22:39:06.978 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:06.978 DEBUG:teuthology.orchestra.run.trial160:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:39:07.035 DEBUG:teuthology.orchestra.run.trial160:> lsblk
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:NAME           MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:nvme2n1        259:0    0   1.7T  0 disk
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:├─vg_nvme-lv_3 253:2    0   700G  0 lvm
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:└─vg_nvme-lv_4 253:3    0   700G  0 lvm
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:nvme0n1        259:1    0 894.3G  0 disk
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:├─nvme0n1p1    259:2    0   512M  0 part /boot/efi
2026-02-20T22:39:07.094 INFO:teuthology.orchestra.run.trial160.stdout:└─nvme0n1p2    259:3    0 893.8G  0 part /
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:nvme1n1        259:4    0   1.7T  0 disk
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:├─vg_nvme-lv_1 253:0    0   700G  0 lvm
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:├─vg_nvme-lv_2 253:1    0   700G  0 lvm
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:└─vg_nvme-lv_5 253:4    0   100G  0 lvm  /var/lib/ceph
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:nvme3n1        259:6    0   700G  0 disk
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:nvme4n1        259:8    0   700G  0 disk
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:nvme5n1        259:10   0   700G  0 disk
2026-02-20T22:39:07.095 INFO:teuthology.orchestra.run.trial160.stdout:nvme6n1        259:12   0   700G  0 disk
2026-02-20T22:39:07.096 DEBUG:teuthology.orchestra.run.trial160:> sudo nvme list -o json
2026-02-20T22:39:07.160 INFO:teuthology.orchestra.run.trial160.stdout:{
2026-02-20T22:39:07.160 INFO:teuthology.orchestra.run.trial160.stdout:  "Devices":[
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme0n1",
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng0n1",
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"E2MU200",
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"Micron_7450_MTFDKBA960TFR",
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"25174FCD5F97",
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":258688479232,
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":1875385008,
2026-02-20T22:39:07.161 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":960197124096,
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme1n1",
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng1n1",
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"E3MQ000",
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"MTFDKCC1T9TGP-1BK1JABYY",
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"092510AB69B9",
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":139589640192,
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":3750748848,
2026-02-20T22:39:07.162 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":1920383410176,
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme2n1",
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng2n1",
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"E3MQ000",
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"MTFDKCC1T9TGP-1BK1JABYY",
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"092510AB699B",
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":139380502528,
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":3750748848,
2026-02-20T22:39:07.163 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":1920383410176,
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme3n1",
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng3n1",
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:07.164 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"861ea7682dabd119f7ac",
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme4n1",
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng4n1",
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:07.165 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"626ccc8886957ac31971",
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme5n1",
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng5n1",
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:07.166 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"581b2612f39fc9008265",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:    },
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:    {
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "NameSpace":1,
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "DevicePath":"/dev/nvme6n1",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "GenericPath":"/dev/ng6n1",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:07.167 INFO:teuthology.orchestra.run.trial160.stdout:      "SerialNumber":"9a30a09e2d9f7bf2c46a",
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:      "SectorSize":512
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:    }
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:  ]
2026-02-20T22:39:07.168 INFO:teuthology.orchestra.run.trial160.stdout:}
2026-02-20T22:39:07.169 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=0 bs=1 count=4096
2026-02-20T22:39:07.237 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.237 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.237 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00517455 s, 792 kB/s
2026-02-20T22:39:07.239 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s0 /dev/nvme3n1
2026-02-20T22:39:07.302 INFO:teuthology.orchestra.run.trial160.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.303 INFO:teuthology.orchestra.run.trial160.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.303 INFO:teuthology.orchestra.run.trial160.stdout:00000016
2026-02-20T22:39:07.304 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:07.370 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.370 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.370 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00438366 s, 934 kB/s
2026-02-20T22:39:07.372 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s1073741824 /dev/nvme3n1
2026-02-20T22:39:07.434 INFO:teuthology.orchestra.run.trial160.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.434 INFO:teuthology.orchestra.run.trial160.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.434 INFO:teuthology.orchestra.run.trial160.stdout:40000016
2026-02-20T22:39:07.435 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:07.505 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.505 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.505 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00497972 s, 823 kB/s
2026-02-20T22:39:07.506 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s10737418240 /dev/nvme3n1
2026-02-20T22:39:07.569 INFO:teuthology.orchestra.run.trial160.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.569 INFO:teuthology.orchestra.run.trial160.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.569 INFO:teuthology.orchestra.run.trial160.stdout:280000016
2026-02-20T22:39:07.570 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=0 bs=1 count=4096
2026-02-20T22:39:07.636 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.637 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.637 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00437668 s, 936 kB/s
2026-02-20T22:39:07.638 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s0 /dev/nvme4n1
2026-02-20T22:39:07.702 INFO:teuthology.orchestra.run.trial160.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.702 INFO:teuthology.orchestra.run.trial160.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.703 INFO:teuthology.orchestra.run.trial160.stdout:00000016
2026-02-20T22:39:07.704 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:07.770 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.771 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.771 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00460977 s, 889 kB/s
2026-02-20T22:39:07.772 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s1073741824 /dev/nvme4n1
2026-02-20T22:39:07.834 INFO:teuthology.orchestra.run.trial160.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.835 INFO:teuthology.orchestra.run.trial160.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.835 INFO:teuthology.orchestra.run.trial160.stdout:40000016
2026-02-20T22:39:07.836 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:07.906 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:07.906 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:07.906 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0048252 s, 849 kB/s
2026-02-20T22:39:07.907 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s10737418240 /dev/nvme4n1
2026-02-20T22:39:07.969 INFO:teuthology.orchestra.run.trial160.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:07.969 INFO:teuthology.orchestra.run.trial160.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:07.969 INFO:teuthology.orchestra.run.trial160.stdout:280000016
2026-02-20T22:39:07.970 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=0 bs=1 count=4096
2026-02-20T22:39:08.038 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.038 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.038 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00437302 s, 937 kB/s
2026-02-20T22:39:08.040 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s0 /dev/nvme5n1
2026-02-20T22:39:08.103 INFO:teuthology.orchestra.run.trial160.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.103 INFO:teuthology.orchestra.run.trial160.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.104 INFO:teuthology.orchestra.run.trial160.stdout:00000016
2026-02-20T22:39:08.105 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:08.172 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.172 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.172 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00461154 s, 888 kB/s
2026-02-20T22:39:08.173 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s1073741824 /dev/nvme5n1
2026-02-20T22:39:08.236 INFO:teuthology.orchestra.run.trial160.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.236 INFO:teuthology.orchestra.run.trial160.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.236 INFO:teuthology.orchestra.run.trial160.stdout:40000016
2026-02-20T22:39:08.237 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:08.305 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.305 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.305 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00445884 s, 919 kB/s
2026-02-20T22:39:08.306 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s10737418240 /dev/nvme5n1
2026-02-20T22:39:08.369 INFO:teuthology.orchestra.run.trial160.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.369 INFO:teuthology.orchestra.run.trial160.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.370 INFO:teuthology.orchestra.run.trial160.stdout:280000016
2026-02-20T22:39:08.371 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=0 bs=1 count=4096
2026-02-20T22:39:08.438 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.438 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.438 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00494091 s, 829 kB/s
2026-02-20T22:39:08.439 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s0 /dev/nvme6n1
2026-02-20T22:39:08.502 INFO:teuthology.orchestra.run.trial160.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.502 INFO:teuthology.orchestra.run.trial160.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.502 INFO:teuthology.orchestra.run.trial160.stdout:00000016
2026-02-20T22:39:08.503 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:08.571 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.571 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.571 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00512593 s, 799 kB/s
2026-02-20T22:39:08.573 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s1073741824 /dev/nvme6n1
2026-02-20T22:39:08.635 INFO:teuthology.orchestra.run.trial160.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.635 INFO:teuthology.orchestra.run.trial160.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.635 INFO:teuthology.orchestra.run.trial160.stdout:40000016
2026-02-20T22:39:08.636 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:08.702 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records in
2026-02-20T22:39:08.702 INFO:teuthology.orchestra.run.trial160.stderr:4096+0 records out
2026-02-20T22:39:08.703 INFO:teuthology.orchestra.run.trial160.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00474057 s, 864 kB/s
2026-02-20T22:39:08.704 DEBUG:teuthology.orchestra.run.trial160:> sudo hexdump -n22 -C -s10737418240 /dev/nvme6n1
2026-02-20T22:39:08.766 INFO:teuthology.orchestra.run.trial160.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:08.766 INFO:teuthology.orchestra.run.trial160.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:08.766 INFO:teuthology.orchestra.run.trial160.stdout:280000016
2026-02-20T22:39:08.768 INFO:tasks.nvme_loop:new_devs ['/dev/nvme3n1', '/dev/nvme4n1', '/dev/nvme5n1', '/dev/nvme6n1']
2026-02-20T22:39:08.768 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:08.768 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/scratch_devs
2026-02-20T22:39:08.831 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:39:08.831 DEBUG:teuthology.orchestra.run.trial192:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:39:08.851 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4']
2026-02-20T22:39:08.851 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/vg_nvme/lv_1
2026-02-20T22:39:08.909 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/vg_nvme/lv_1 -> ../dm-0
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1305        Links: 1
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:39:05.456035121 +0000
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:05.325036605 +0000
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:05.325036605 +0000
2026-02-20T22:39:08.910 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:05.325036605 +0000
2026-02-20T22:39:08.911 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1
2026-02-20T22:39:08.975 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:39:08.975 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:39:08.975 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000178211 s, 2.9 MB/s
2026-02-20T22:39:08.977 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1
2026-02-20T22:39:09.034 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/vg_nvme/lv_2
2026-02-20T22:39:09.092 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/vg_nvme/lv_2 -> ../dm-1
2026-02-20T22:39:09.092 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:09.092 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1302        Links: 1
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:39:05.456035121 +0000
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.093 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.093 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1
2026-02-20T22:39:09.157 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:39:09.157 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:39:09.157 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000171821 s, 3.0 MB/s
2026-02-20T22:39:09.158 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2
2026-02-20T22:39:09.215 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/vg_nvme/lv_3
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/vg_nvme/lv_3 -> ../dm-2
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1296        Links: 1
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:39:05.456035121 +0000
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.272 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.273 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1
2026-02-20T22:39:09.336 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:39:09.336 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:39:09.336 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.00014557 s, 3.5 MB/s
2026-02-20T22:39:09.337 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3
2026-02-20T22:39:09.396 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/vg_nvme/lv_4
2026-02-20T22:39:09.452 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/vg_nvme/lv_4 -> ../dm-3
2026-02-20T22:39:09.452 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
2026-02-20T22:39:09.452 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1292        Links: 1
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:device_t:s0
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:39:05.457035110 +0000
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.453 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:05.324036616 +0000
2026-02-20T22:39:09.453 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1
2026-02-20T22:39:09.516 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:39:09.516 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:39:09.517 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000145411 s, 3.5 MB/s
2026-02-20T22:39:09.518 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4
2026-02-20T22:39:09.577 DEBUG:teuthology.orchestra.run.trial192:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype
2026-02-20T22:39:09.682 INFO:teuthology.orchestra.run.trial192.stdout:loop
2026-02-20T22:39:09.684 INFO:tasks.nvme_loop:Connecting nvme_loop trial192:/dev/vg_nvme/lv_1...
2026-02-20T22:39:09.684 DEBUG:teuthology.orchestra.run.trial192:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn
2026-02-20T22:39:09.716 INFO:teuthology.orchestra.run.trial192.stdout:1
2026-02-20T22:39:09.742 INFO:teuthology.orchestra.run.trial192.stdout:/dev/vg_nvme/lv_11
2026-02-20T22:39:09.777 INFO:teuthology.orchestra.run.trial192.stdout:connecting to device: nvme3
2026-02-20T22:39:09.779 INFO:tasks.nvme_loop:Connecting nvme_loop trial192:/dev/vg_nvme/lv_2...
2026-02-20T22:39:09.779 DEBUG:teuthology.orchestra.run.trial192:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn
2026-02-20T22:39:09.852 INFO:teuthology.orchestra.run.trial192.stdout:1
2026-02-20T22:39:09.883 INFO:teuthology.orchestra.run.trial192.stdout:/dev/vg_nvme/lv_21
2026-02-20T22:39:09.912 INFO:teuthology.orchestra.run.trial192.stdout:connecting to device: nvme4
2026-02-20T22:39:09.914 INFO:tasks.nvme_loop:Connecting nvme_loop trial192:/dev/vg_nvme/lv_3...
2026-02-20T22:39:09.915 DEBUG:teuthology.orchestra.run.trial192:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn
2026-02-20T22:39:09.986 INFO:teuthology.orchestra.run.trial192.stdout:1
2026-02-20T22:39:10.012 INFO:teuthology.orchestra.run.trial192.stdout:/dev/vg_nvme/lv_31
2026-02-20T22:39:10.046 INFO:teuthology.orchestra.run.trial192.stdout:connecting to device: nvme5
2026-02-20T22:39:10.048 INFO:tasks.nvme_loop:Connecting nvme_loop trial192:/dev/vg_nvme/lv_4...
2026-02-20T22:39:10.048 DEBUG:teuthology.orchestra.run.trial192:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn
2026-02-20T22:39:10.120 INFO:teuthology.orchestra.run.trial192.stdout:1
2026-02-20T22:39:10.149 INFO:teuthology.orchestra.run.trial192.stdout:/dev/vg_nvme/lv_41
2026-02-20T22:39:10.184 INFO:teuthology.orchestra.run.trial192.stdout:connecting to device: nvme6
2026-02-20T22:39:10.185 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:39:10.185 DEBUG:teuthology.orchestra.run.trial192:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:39:10.241 DEBUG:teuthology.orchestra.run.trial192:> lsblk
2026-02-20T22:39:10.300 INFO:teuthology.orchestra.run.trial192.stdout:NAME           MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
2026-02-20T22:39:10.300 INFO:teuthology.orchestra.run.trial192.stdout:nvme2n1        259:0    0   1.7T  0 disk
2026-02-20T22:39:10.300 INFO:teuthology.orchestra.run.trial192.stdout:├─vg_nvme-lv_3 253:2    0   700G  0 lvm
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:└─vg_nvme-lv_4 253:3    0   700G  0 lvm
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme1n1        259:1    0   1.7T  0 disk
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:├─vg_nvme-lv_1 253:0    0   700G  0 lvm
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:├─vg_nvme-lv_2 253:1    0   700G  0 lvm
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:└─vg_nvme-lv_5 253:4    0   100G  0 lvm  /var/lib/ceph
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme0n1        259:2    0 894.3G  0 disk
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:├─nvme0n1p1    259:3    0   512M  0 part /boot/efi
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:└─nvme0n1p2    259:4    0 893.8G  0 part /
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme3n1        259:6    0   700G  0 disk
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme4n1        259:8    0   700G  0 disk
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme5n1        259:10   0   700G  0 disk
2026-02-20T22:39:10.301 INFO:teuthology.orchestra.run.trial192.stdout:nvme6n1        259:12   0   700G  0 disk
2026-02-20T22:39:10.302 DEBUG:teuthology.orchestra.run.trial192:> sudo nvme list -o json
2026-02-20T22:39:10.366 INFO:teuthology.orchestra.run.trial192.stdout:{
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:  "Devices":[
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme0n1",
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng0n1",
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"E2MU200",
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"Micron_7450_MTFDKBA960TFR",
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"25174FCD5EB1",
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":447030726656,
2026-02-20T22:39:10.367 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":1875385008,
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":960197124096,
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme1n1",
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng1n1",
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"E3MQ000",
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"MTFDKCC1T9TGP-1BK1JABYY",
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"012510B00295",
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":140830208000,
2026-02-20T22:39:10.368 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":3750748848,
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":1920383410176,
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme2n1",
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng2n1",
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"E3MQ000",
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"MTFDKCC1T9TGP-1BK1JABYY",
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"012510B002AA",
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":141471236096,
2026-02-20T22:39:10.369 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":3750748848,
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":1920383410176,
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme3n1",
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng3n1",
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"9224ef463e42b9247c66",
2026-02-20T22:39:10.370 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme4n1",
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng4n1",
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:10.371 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"da8942577159d4ebd296",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme5n1",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng5n1",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:10.372 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"48bbdeb254abff1a7b96",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:    },
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:    {
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "NameSpace":1,
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "DevicePath":"/dev/nvme6n1",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "GenericPath":"/dev/ng6n1",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "Firmware":"5.14.0-6",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "ModelNumber":"Linux",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "SerialNumber":"f0d505abb275dbf36d23",
2026-02-20T22:39:10.373 INFO:teuthology.orchestra.run.trial192.stdout:      "UsedBytes":751619276800,
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:      "MaximumLBA":1468006400,
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:      "PhysicalSize":751619276800,
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:      "SectorSize":512
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:    }
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:  ]
2026-02-20T22:39:10.374 INFO:teuthology.orchestra.run.trial192.stdout:}
2026-02-20T22:39:10.375 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=0 bs=1 count=4096
2026-02-20T22:39:10.444 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:10.444 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:10.444 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00443756 s, 923 kB/s
2026-02-20T22:39:10.445 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s0 /dev/nvme3n1
2026-02-20T22:39:10.508 INFO:teuthology.orchestra.run.trial192.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:10.508 INFO:teuthology.orchestra.run.trial192.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:10.508 INFO:teuthology.orchestra.run.trial192.stdout:00000016
2026-02-20T22:39:10.509 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:10.576 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:10.576 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:10.576 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00486939 s, 841 kB/s
2026-02-20T22:39:10.578 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s1073741824 /dev/nvme3n1
2026-02-20T22:39:10.642 INFO:teuthology.orchestra.run.trial192.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:10.642 INFO:teuthology.orchestra.run.trial192.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:10.642 INFO:teuthology.orchestra.run.trial192.stdout:40000016
2026-02-20T22:39:10.643 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme3n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:10.710 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:10.710 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:10.710 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0044025 s, 930 kB/s
2026-02-20T22:39:10.712 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s10737418240 /dev/nvme3n1
2026-02-20T22:39:10.774 INFO:teuthology.orchestra.run.trial192.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:10.775 INFO:teuthology.orchestra.run.trial192.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:10.775 INFO:teuthology.orchestra.run.trial192.stdout:280000016
2026-02-20T22:39:10.776 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=0 bs=1 count=4096
2026-02-20T22:39:10.842 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:10.843 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:10.843 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00438978 s, 933 kB/s
2026-02-20T22:39:10.844 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s0 /dev/nvme4n1
2026-02-20T22:39:10.906 INFO:teuthology.orchestra.run.trial192.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:10.907 INFO:teuthology.orchestra.run.trial192.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:10.907 INFO:teuthology.orchestra.run.trial192.stdout:00000016
2026-02-20T22:39:10.908 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:10.978 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:10.978 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:10.978 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00521817 s, 785 kB/s
2026-02-20T22:39:10.979 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s1073741824 /dev/nvme4n1
2026-02-20T22:39:11.042 INFO:teuthology.orchestra.run.trial192.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.042 INFO:teuthology.orchestra.run.trial192.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.042 INFO:teuthology.orchestra.run.trial192.stdout:40000016
2026-02-20T22:39:11.043 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme4n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:11.111 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.111 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.111 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00491215 s, 834 kB/s
2026-02-20T22:39:11.112 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s10737418240 /dev/nvme4n1
2026-02-20T22:39:11.175 INFO:teuthology.orchestra.run.trial192.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.175 INFO:teuthology.orchestra.run.trial192.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.175 INFO:teuthology.orchestra.run.trial192.stdout:280000016
2026-02-20T22:39:11.176 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=0 bs=1 count=4096
2026-02-20T22:39:11.244 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.245 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.245 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00441092 s, 929 kB/s
2026-02-20T22:39:11.247 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s0 /dev/nvme5n1
2026-02-20T22:39:11.310 INFO:teuthology.orchestra.run.trial192.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.310 INFO:teuthology.orchestra.run.trial192.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.310 INFO:teuthology.orchestra.run.trial192.stdout:00000016
2026-02-20T22:39:11.311 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:11.378 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.378 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.378 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00444555 s, 921 kB/s
2026-02-20T22:39:11.379 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s1073741824 /dev/nvme5n1
2026-02-20T22:39:11.442 INFO:teuthology.orchestra.run.trial192.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.442 INFO:teuthology.orchestra.run.trial192.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.443 INFO:teuthology.orchestra.run.trial192.stdout:40000016
2026-02-20T22:39:11.444 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme5n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:11.511 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.512 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.512 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00481367 s, 851 kB/s
2026-02-20T22:39:11.513 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s10737418240 /dev/nvme5n1
2026-02-20T22:39:11.577 INFO:teuthology.orchestra.run.trial192.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.577 INFO:teuthology.orchestra.run.trial192.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.577 INFO:teuthology.orchestra.run.trial192.stdout:280000016
2026-02-20T22:39:11.578 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=0 bs=1 count=4096
2026-02-20T22:39:11.645 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.646 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.646 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00474244 s, 864 kB/s
2026-02-20T22:39:11.647 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s0 /dev/nvme6n1
2026-02-20T22:39:11.711 INFO:teuthology.orchestra.run.trial192.stdout:00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.712 INFO:teuthology.orchestra.run.trial192.stdout:00000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.712 INFO:teuthology.orchestra.run.trial192.stdout:00000016
2026-02-20T22:39:11.713 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=1073741824 bs=1 count=4096
2026-02-20T22:39:11.780 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.780 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.781 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00467577 s, 876 kB/s
2026-02-20T22:39:11.782 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s1073741824 /dev/nvme6n1
2026-02-20T22:39:11.845 INFO:teuthology.orchestra.run.trial192.stdout:40000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.845 INFO:teuthology.orchestra.run.trial192.stdout:40000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.845 INFO:teuthology.orchestra.run.trial192.stdout:40000016
2026-02-20T22:39:11.846 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/zero of=/dev/nvme6n1 seek=10737418240 bs=1 count=4096
2026-02-20T22:39:11.913 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records in
2026-02-20T22:39:11.914 INFO:teuthology.orchestra.run.trial192.stderr:4096+0 records out
2026-02-20T22:39:11.914 INFO:teuthology.orchestra.run.trial192.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00476169 s, 860 kB/s
2026-02-20T22:39:11.915 DEBUG:teuthology.orchestra.run.trial192:> sudo hexdump -n22 -C -s10737418240 /dev/nvme6n1
2026-02-20T22:39:11.977 INFO:teuthology.orchestra.run.trial192.stdout:280000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
2026-02-20T22:39:11.978 INFO:teuthology.orchestra.run.trial192.stdout:280000010  00 00 00 00 00 00                                 |......|
2026-02-20T22:39:11.978 INFO:teuthology.orchestra.run.trial192.stdout:280000016
2026-02-20T22:39:11.979 INFO:tasks.nvme_loop:new_devs ['/dev/nvme3n1', '/dev/nvme4n1', '/dev/nvme5n1', '/dev/nvme6n1']
2026-02-20T22:39:11.979 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:39:11.979 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/scratch_devs
2026-02-20T22:39:12.043 INFO:teuthology.run_tasks:Running task cephadm...
2026-02-20T22:39:12.172 INFO:tasks.cephadm:Config: {'roleless': True, 'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_DAEMON_PLACE_FAIL', 'CEPHADM_FAILED_DAEMON'], 'log-only-match': ['CEPHADM_'], 'sha1': 'f7870454149f22574086dd2ff55d050585136531'}
2026-02-20T22:39:12.172 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531
2026-02-20T22:39:12.174 INFO:tasks.cephadm:Cluster fsid is fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:12.174 INFO:tasks.cephadm:Choosing monitor IPs and ports...
2026-02-20T22:39:12.175 INFO:tasks.cephadm:No mon roles; fabricating mons
2026-02-20T22:39:12.175 INFO:tasks.cephadm:Monitor IPs: {'mon.trial160': '10.20.193.160', 'mon.trial192': '10.20.193.192'}
2026-02-20T22:39:12.175 INFO:tasks.cephadm:Normalizing hostnames...
2026-02-20T22:39:12.175 DEBUG:teuthology.orchestra.run.trial160:> sudo hostname $(hostname -s)
2026-02-20T22:39:12.201 DEBUG:teuthology.orchestra.run.trial192:> sudo hostname $(hostname -s)
2026-02-20T22:39:12.226 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra
2026-02-20T22:39:12.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f7870454149f22574086dd2ff55d050585136531
2026-02-20T22:39:12.372 INFO:tasks.cephadm:builder_project result: [{'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/flavors/default/', 'ref': 'wip-pdonnell-testing-20260219.182737-tentacle', 'sha1': 'f7870454149f22574086dd2ff55d050585136531', 'distro': 'centos', 'distro_version': '9', 'distro_codename': None, 'modified': '2026-02-20 13:26:37.513007', 'status': 'ready', 'flavor': 'default', 'project': 'ceph', 'archs': ['source', 'x86_64'], 'extra': {'version': '20.2.0-677-gf7870454', 'package_manager_version': '20.2.0-677.gf7870454', 'build_url': 'https://jenkins.ceph.com/job/ceph-dev-pipeline/3114/', 'root_build_cause': '', 'node_name': '10.20.192.18+soko08', 'job_name': 'ceph-dev-pipeline'}}]
2026-02-20T22:39:12.491 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-pdonnell-testing-20260219.182737-tentacle, sha1 f7870454149f22574086dd2ff55d050585136531 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=f7870454149f22574086dd2ff55d050585136531
2026-02-20T22:39:12.492 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/x86_64/flavors/default/cephadm
2026-02-20T22:39:12.493 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/x86_64/flavors/default/cephadm
2026-02-20T22:39:12.493 DEBUG:teuthology.orchestra.run.trial160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:12.847 INFO:teuthology.orchestra.run.trial160.stdout:-rw-r--r--. 1 ubuntu ubuntu 1054032 Feb 20 22:39 /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:12.848 DEBUG:teuthology.orchestra.run.trial192:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20260219.182737-tentacle/f7870454149f22574086dd2ff55d050585136531/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:13.191 INFO:teuthology.orchestra.run.trial192.stdout:-rw-r--r--. 1 ubuntu ubuntu 1054032 Feb 20 22:39 /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:13.192 DEBUG:teuthology.orchestra.run.trial160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:13.211 DEBUG:teuthology.orchestra.run.trial192:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm
2026-02-20T22:39:13.236 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 on all hosts...
2026-02-20T22:39:13.236 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 pull
2026-02-20T22:39:13.253 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 pull
2026-02-20T22:39:13.497 INFO:teuthology.orchestra.run.trial160.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531...
2026-02-20T22:39:13.509 INFO:teuthology.orchestra.run.trial192.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531...
2026-02-20T22:39:22.339 INFO:teuthology.orchestra.run.trial160.stdout:{
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:    "ceph_version": "ceph version 20.2.0-677-gf7870454 (f7870454149f22574086dd2ff55d050585136531) tentacle (stable)",
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:    "image_id": "191e36ff62c2c47de9f007d502ed6fbffff443749f925d00e4709abf526900f1",
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:    "repo_digests": [
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:        "quay.ceph.io/ceph-ci/ceph@sha256:9cc281dd1e60310f8109166bc76e656ce4c5c9b7d9bb9abfc9aa033dfa762ae4"
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:    ]
2026-02-20T22:39:22.340 INFO:teuthology.orchestra.run.trial160.stdout:}
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:{
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:    "ceph_version": "ceph version 20.2.0-677-gf7870454 (f7870454149f22574086dd2ff55d050585136531) tentacle (stable)",
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:    "image_id": "191e36ff62c2c47de9f007d502ed6fbffff443749f925d00e4709abf526900f1",
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:    "repo_digests": [
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:        "quay.ceph.io/ceph-ci/ceph@sha256:9cc281dd1e60310f8109166bc76e656ce4c5c9b7d9bb9abfc9aa033dfa762ae4"
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:    ]
2026-02-20T22:39:22.350 INFO:teuthology.orchestra.run.trial192.stdout:}
2026-02-20T22:39:22.367 DEBUG:teuthology.orchestra.run.trial160:> sudo mkdir -p /etc/ceph
2026-02-20T22:39:22.428 DEBUG:teuthology.orchestra.run.trial192:> sudo mkdir -p /etc/ceph
2026-02-20T22:39:22.454 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 777 /etc/ceph
2026-02-20T22:39:22.478 DEBUG:teuthology.orchestra.run.trial192:> sudo chmod 777 /etc/ceph
2026-02-20T22:39:22.517 INFO:tasks.cephadm:Writing seed config...
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [mgr] debug mgr = 20
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [mgr] debug ms = 1
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [mon] debug mon = 20
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [mon] debug ms = 1
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [mon] debug paxos = 20
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [osd] debug ms = 1
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [osd] debug osd = 20
2026-02-20T22:39:22.518 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True
2026-02-20T22:39:22.519 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:22.519 DEBUG:teuthology.orchestra.run.trial160:> dd of=/home/ubuntu/cephtest/seed.ceph.conf
2026-02-20T22:39:22.534 DEBUG:tasks.cephadm:Final config:
[global]
# make logging friendly to teuthology
log_to_file = true
log_to_stderr = false
log to journald = false
mon cluster log to file = true
mon cluster log file level = debug

mon clock drift allowed = 1.000

# replicate across OSDs, not hosts
osd crush chooseleaf type = 0
#osd pool default size = 2
osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd

# enable some debugging
auth debug = true
ms die on old message = true
ms die on bug = true
debug asserts on shutdown = true

# adjust warnings
mon max pg per osd = 10000# >= luminous
mon pg warn max object skew = 0
mon osd allow primary affinity = true
mon osd allow pg remap = true
mon warn on legacy crush tunables = false
mon warn on crush straw calc version zero = false
mon warn on no sortbitwise = false
mon warn on osd down out interval zero = false
mon warn on too few osds = false
mon_warn_on_pool_pg_num_not_power_of_two = false

# disable pg_autoscaler by default for new pools
osd_pool_default_pg_autoscale_mode = off

# tests delete pools
mon allow pool delete = true
fsid = fa4d44c0-0eac-11f1-86c6-d404e6e7d460

[osd]
osd scrub load threshold = 5.0
osd scrub max interval = 600
osd mclock profile = high_recovery_ops
osd mclock skip benchmark = true

osd recover clone overlap = true
osd recovery max chunk = 1048576

osd deep scrub update digest min age = 30

osd map max advance = 10

osd memory target autotune = true

# debugging
osd debug shutdown = true
osd debug op order = true
osd debug verify stray on activate = true
osd debug pg log writeout = true
osd debug verify cached snaps = true
osd debug verify missing on start = true
osd debug misdirected ops = true
osd op queue = debug_random
osd op queue cut off = debug_random
osd shutdown pgref assert = True
bdev debug aio = true
osd sloppy crc = true
debug ms = 1
debug osd = 20

[mgr]
mon reweight min pgs per osd = 4
mon reweight min bytes per osd = 10
mgr/telemetry/nag = false
debug mgr = 20
debug ms = 1

[mon]
mon data avail warn = 5
mon mgr mkfs grace = 240
mon reweight min pgs per osd = 4
mon osd reporter subtree level = osd
mon osd prime pg temp = true
mon reweight min bytes per osd = 10

# rotate auth tickets quickly to exercise renewal paths
auth mon ticket ttl = 660# 11m
auth service ticket ttl = 240# 4m

# don't complain about global id reclaim
mon_warn_on_insecure_global_id_reclaim = false
mon_warn_on_insecure_global_id_reclaim_allowed = false
debug mon = 20
debug ms = 1
debug paxos = 20

[client.rgw]
rgw cache enabled = true
rgw enable ops log = true
rgw enable usage log = true

2026-02-20T22:39:22.534 DEBUG:teuthology.orchestra.run.trial160:mon.trial160> sudo journalctl -f -n 0 -u ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160.service
2026-02-20T22:39:22.576 INFO:tasks.cephadm:Bootstrapping...
2026-02-20T22:39:22.576 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 -v bootstrap --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 10.20.193.160 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:39:22.714 INFO:teuthology.orchestra.run.trial160.stdout:--------------------------------------------------------------------------------
2026-02-20T22:39:22.714 INFO:teuthology.orchestra.run.trial160.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531', '-v', 'bootstrap', '--fsid', 'fa4d44c0-0eac-11f1-86c6-d404e6e7d460', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '10.20.193.160', '--skip-admin-label']
2026-02-20T22:39:22.714 INFO:teuthology.orchestra.run.trial160.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts.
2026-02-20T22:39:22.714 INFO:teuthology.orchestra.run.trial160.stdout:Verifying podman|docker is present...
2026-02-20T22:39:22.732 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stdout 5.6.0
2026-02-20T22:39:22.732 INFO:teuthology.orchestra.run.trial160.stdout:Verifying lvm2 is present...
2026-02-20T22:39:22.732 INFO:teuthology.orchestra.run.trial160.stdout:Verifying time synchronization is in place...
2026-02-20T22:39:22.738 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service
2026-02-20T22:39:22.738 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory
2026-02-20T22:39:22.744 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service
2026-02-20T22:39:22.744 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:22.750 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout enabled
2026-02-20T22:39:22.756 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout active
2026-02-20T22:39:22.756 INFO:teuthology.orchestra.run.trial160.stdout:Unit chronyd.service is enabled and running
2026-02-20T22:39:22.756 INFO:teuthology.orchestra.run.trial160.stdout:Repeating the final host check...
2026-02-20T22:39:22.771 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stdout 5.6.0
2026-02-20T22:39:22.771 INFO:teuthology.orchestra.run.trial160.stdout:podman (/bin/podman) version 5.6.0 is present
2026-02-20T22:39:22.771 INFO:teuthology.orchestra.run.trial160.stdout:systemctl is present
2026-02-20T22:39:22.771 INFO:teuthology.orchestra.run.trial160.stdout:lvcreate is present
2026-02-20T22:39:22.777 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service
2026-02-20T22:39:22.777 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory
2026-02-20T22:39:22.782 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active chrony.service
2026-02-20T22:39:22.782 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:22.787 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout enabled
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout active
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:Unit chronyd.service is enabled and running
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:Host looks OK
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:Cluster fsid: fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:Acquiring lock 140014949744896 on /run/cephadm/fa4d44c0-0eac-11f1-86c6-d404e6e7d460.lock
2026-02-20T22:39:22.793 INFO:teuthology.orchestra.run.trial160.stdout:Lock 140014949744896 acquired on /run/cephadm/fa4d44c0-0eac-11f1-86c6-d404e6e7d460.lock
2026-02-20T22:39:22.794 INFO:teuthology.orchestra.run.trial160.stdout:Verifying IP 10.20.193.160 port 3300 ...
2026-02-20T22:39:22.794 INFO:teuthology.orchestra.run.trial160.stdout:Verifying IP 10.20.193.160 port 6789 ...
2026-02-20T22:39:22.794 INFO:teuthology.orchestra.run.trial160.stdout:Base mon IP(s) is [10.20.193.160:3300, 10.20.193.160:6789], mon addrv is [v2:10.20.193.160:3300,v1:10.20.193.160:6789]
2026-02-20T22:39:22.797 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout default via 10.20.192.1 dev enp1s0f0 proto dhcp src 10.20.193.160 metric 102
2026-02-20T22:39:22.797 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout 10.20.192.0/20 dev enp1s0f0 proto kernel scope link src 10.20.193.160 metric 102
2026-02-20T22:39:22.797 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout 169.254.3.0/24 dev enp7s0f4u2u2c2 proto kernel scope link src 169.254.3.1 metric 101
2026-02-20T22:39:22.800 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium
2026-02-20T22:39:22.800 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout fe80::/64 dev enp1s0f0 proto kernel metric 256 pref medium
2026-02-20T22:39:22.800 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout fe80::/64 dev enp7s0f4u2u2c2 proto kernel metric 1024 pref medium
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout     inet6 ::1/128 scope host
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout        valid_lft forever preferred_lft forever
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout 2: enp7s0f4u2u2c2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout     inet6 fe80::473:9523:996c:f669/64 scope link noprefixroute
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout        valid_lft forever preferred_lft forever
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout 3: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout     inet6 fe80::925a:8ff:fe77:634e/64 scope link
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:/sbin/ip: stdout        valid_lft forever preferred_lft forever
2026-02-20T22:39:22.803 INFO:teuthology.orchestra.run.trial160.stdout:Mon IP `10.20.193.160` is in CIDR network `10.20.192.0/20`
2026-02-20T22:39:22.804 INFO:teuthology.orchestra.run.trial160.stdout:Mon IP `10.20.193.160` is in CIDR network `10.20.192.0/20`
2026-02-20T22:39:22.804 INFO:teuthology.orchestra.run.trial160.stdout:Inferred mon public CIDR from local network configuration ['10.20.192.0/20', '10.20.192.0/20']
2026-02-20T22:39:22.804 INFO:teuthology.orchestra.run.trial160.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network
2026-02-20T22:39:22.804 INFO:teuthology.orchestra.run.trial160.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531...
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stdout 191e36ff62c2c47de9f007d502ed6fbffff443749f925d00e4709abf526900f1
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531...
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Getting image source signatures
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Copying blob sha256:0cf63a27055a859d286185a933b0b6ddb181f8349d6aee8c52cf789385a51e06
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Copying blob sha256:5dd85e1a50663785d86adae5ef92579a03e6f64059ca9c005010518acdd62048
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Copying config sha256:191e36ff62c2c47de9f007d502ed6fbffff443749f925d00e4709abf526900f1
2026-02-20T22:39:23.041 INFO:teuthology.orchestra.run.trial160.stdout:/bin/podman: stderr Writing manifest to image destination
2026-02-20T22:39:23.182 INFO:teuthology.orchestra.run.trial160.stdout:ceph: stdout ceph version 20.2.0-677-gf7870454 (f7870454149f22574086dd2ff55d050585136531) tentacle (stable)
2026-02-20T22:39:23.182 INFO:teuthology.orchestra.run.trial160.stdout:Ceph version: ceph version 20.2.0-677-gf7870454 (f7870454149f22574086dd2ff55d050585136531) tentacle (stable)
2026-02-20T22:39:23.182 INFO:teuthology.orchestra.run.trial160.stdout:Extracting ceph user uid/gid from container image...
2026-02-20T22:39:23.302 INFO:teuthology.orchestra.run.trial160.stdout:stat: stdout 167 167
2026-02-20T22:39:23.303 INFO:teuthology.orchestra.run.trial160.stdout:Creating initial keys...
2026-02-20T22:39:23.431 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph-authtool: stdout AQCb4phpX6nmGBAAn7562tpM60LqcD2+qi7xfQ==
2026-02-20T22:39:23.564 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph-authtool: stdout AQCb4phpIcjzIBAA0ulW5zT7raj2iQKUKogI5w==
2026-02-20T22:39:23.692 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph-authtool: stdout AQCb4phpc5CDKBAAVHadjTUmjBHqcGE5BgQQag==
2026-02-20T22:39:23.693 INFO:teuthology.orchestra.run.trial160.stdout:Creating initial monmap...
2026-02-20T22:39:23.833 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap
2026-02-20T22:39:23.833 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = tentacle
2026-02-20T22:39:23.833 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors)
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:monmaptool for trial160 [v2:10.20.193.160:3300,v1:10.20.193.160:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:setting min_mon_release = tentacle
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: set fsid to fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors)
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:23.834 INFO:teuthology.orchestra.run.trial160.stdout:Creating mon...
2026-02-20T22:39:23.960 INFO:teuthology.orchestra.run.trial160.stdout:create mon.trial160 on
2026-02-20T22:39:24.247 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target.
2026-02-20T22:39:24.356 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target → /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target.
2026-02-20T22:39:24.357 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target → /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target.
2026-02-20T22:39:24.491 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160
2026-02-20T22:39:24.491 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160.service: Unit ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160.service not loaded.
2026-02-20T22:39:24.621 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target.wants/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160.service → /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@.service.
2026-02-20T22:39:24.765 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service
2026-02-20T22:39:24.766 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout disabled
2026-02-20T22:39:24.771 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service
2026-02-20T22:39:24.771 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:24.771 INFO:teuthology.orchestra.run.trial160.stdout:firewalld.service is not enabled
2026-02-20T22:39:24.771 INFO:teuthology.orchestra.run.trial160.stdout:Not possible to enable service <ceph-mon>. firewalld.service is not available
2026-02-20T22:39:24.772 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mon to start...
2026-02-20T22:39:24.772 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mon...
2026-02-20T22:39:24.932 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout   cluster:
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     id:     fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     health: HEALTH_OK
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout   services:
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     mon: 1 daemons, quorum trial160 (age 0.135821s) [leader: trial160]
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     mgr: no daemons active
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     osd: 0 osds: 0 up, 0 in
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:24.933 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout   data:
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     pools:   0 pools, 0 pgs
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     objects: 0 objects, 0 B
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     usage:   0 B used, 0 B / 0 B avail
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     pgs:
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:mon is available
2026-02-20T22:39:24.934 INFO:teuthology.orchestra.run.trial160.stdout:Assimilating anything we can from ceph.conf...
2026-02-20T22:39:25.085 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:25.085 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [global]
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	fsid = fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_cluster_log_file_level = debug
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_host = [v2:10.20.193.160:3300,v1:10.20.193.160:6789]
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_osd_allow_pg_remap = true
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_osd_allow_primary_affinity = true
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_warn_on_no_sortbitwise = false
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_crush_chooseleaf_type = 0
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [mgr]
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mgr/telemetry/nag = false
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [osd]
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_map_max_advance = 10
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_sloppy_crc = true
2026-02-20T22:39:25.086 INFO:teuthology.orchestra.run.trial160.stdout:Generating new minimal ceph.conf...
2026-02-20T22:39:25.239 INFO:teuthology.orchestra.run.trial160.stdout:Restarting the monitor...
2026-02-20T22:39:25.617 INFO:teuthology.orchestra.run.trial160.stdout:Setting public_network to 10.20.192.0/20 in global config section
2026-02-20T22:39:25.774 INFO:teuthology.orchestra.run.trial160.stdout:Wrote config to /etc/ceph/ceph.conf
2026-02-20T22:39:25.775 INFO:teuthology.orchestra.run.trial160.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:39:25.775 INFO:teuthology.orchestra.run.trial160.stdout:Creating mgr...
2026-02-20T22:39:25.775 INFO:teuthology.orchestra.run.trial160.stdout:Verifying port 0.0.0.0:9283 ...
2026-02-20T22:39:25.776 INFO:teuthology.orchestra.run.trial160.stdout:Verifying port 0.0.0.0:8765 ...
2026-02-20T22:39:25.776 INFO:teuthology.orchestra.run.trial160.stdout:Verifying port 0.0.0.0:8443 ...
2026-02-20T22:39:25.912 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mgr.trial160.pdmeos
2026-02-20T22:39:25.913 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Failed to reset failed state of unit ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mgr.trial160.pdmeos.service: Unit ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mgr.trial160.pdmeos.service not loaded.
2026-02-20T22:39:26.030 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460.target.wants/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mgr.trial160.pdmeos.service → /etc/systemd/system/ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@.service.
2026-02-20T22:39:26.174 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service
2026-02-20T22:39:26.174 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout disabled
2026-02-20T22:39:26.180 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service
2026-02-20T22:39:26.181 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:26.181 INFO:teuthology.orchestra.run.trial160.stdout:firewalld.service is not enabled
2026-02-20T22:39:26.181 INFO:teuthology.orchestra.run.trial160.stdout:Not possible to enable service <ceph>. firewalld.service is not available
2026-02-20T22:39:26.187 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service
2026-02-20T22:39:26.188 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout disabled
2026-02-20T22:39:26.194 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service
2026-02-20T22:39:26.194 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:26.194 INFO:teuthology.orchestra.run.trial160.stdout:firewalld.service is not enabled
2026-02-20T22:39:26.195 INFO:teuthology.orchestra.run.trial160.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available
2026-02-20T22:39:26.195 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mgr to start...
2026-02-20T22:39:26.195 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mgr...
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsid": "fa4d44c0-0eac-11f1-86c6-d404e6e7d460",
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "health": {
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "status": "HEALTH_OK",
2026-02-20T22:39:26.362 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "checks": {},
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "mutes": []
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "election_epoch": 5,
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum": [
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         0
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_names": [
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "trial160"
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_age": 0,
2026-02-20T22:39:26.363 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "monmap": {
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "min_mon_release_name": "tentacle",
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_mons": 1
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "osdmap": {
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_osds": 0,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_up_osds": 0,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_up_since": 0,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_in_osds": 0,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_in_since": 0,
2026-02-20T22:39:26.364 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_remapped_pgs": 0
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "pgmap": {
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "pgs_by_state": [],
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pgs": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pools": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_objects": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "data_bytes": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_used": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_avail": 0,
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_total": 0
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.365 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsmap": {
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "btime": "2026-02-20T22:39:24:775392+0000",
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "by_rank": [],
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "up:standby": 0
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "mgrmap": {
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "available": false,
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_standbys": 0,
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modules": [
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "iostat",
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "nfs"
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         ],
2026-02-20T22:39:26.366 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "servicemap": {
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modified": "2026-02-20T22:39:24.775772+0000",
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "progress_events": {}
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:26.367 INFO:teuthology.orchestra.run.trial160.stdout:mgr not available, waiting (1/15)...
2026-02-20T22:39:27.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:26 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2190956885' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsid": "fa4d44c0-0eac-11f1-86c6-d404e6e7d460",
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "health": {
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "status": "HEALTH_OK",
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "checks": {},
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "mutes": []
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.520 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "election_epoch": 5,
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum": [
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         0
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_names": [
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "trial160"
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_age": 2,
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "monmap": {
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "min_mon_release_name": "tentacle",
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_mons": 1
2026-02-20T22:39:28.521 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "osdmap": {
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_osds": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_up_osds": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_up_since": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_in_osds": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_in_since": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_remapped_pgs": 0
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "pgmap": {
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "pgs_by_state": [],
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pgs": 0,
2026-02-20T22:39:28.522 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pools": 0,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_objects": 0,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "data_bytes": 0,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_used": 0,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_avail": 0,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_total": 0
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsmap": {
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "btime": "2026-02-20T22:39:24:775392+0000",
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "by_rank": [],
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "up:standby": 0
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.523 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "mgrmap": {
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "available": false,
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_standbys": 0,
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modules": [
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "iostat",
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "nfs"
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         ],
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "servicemap": {
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modified": "2026-02-20T22:39:24.775772+0000",
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:28.524 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:28.525 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "progress_events": {}
2026-02-20T22:39:28.525 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:28.525 INFO:teuthology.orchestra.run.trial160.stdout:mgr not available, waiting (2/15)...
2026-02-20T22:39:28.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3326597589' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: Activating manager daemon trial160.pdmeos
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: mgrmap e2: trial160.pdmeos(active, starting, since 0.00239173s)
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:39:29.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:39:29.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:39:29.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:39:29.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:29.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:29.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:29 trial160 ceph-mon[35528]: from='mgr.14100 10.20.193.160:0/3490656590' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:30.788 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:30.788 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:30.788 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsid": "fa4d44c0-0eac-11f1-86c6-d404e6e7d460",
2026-02-20T22:39:30.788 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "health": {
2026-02-20T22:39:30.788 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "status": "HEALTH_OK",
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "checks": {},
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "mutes": []
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "election_epoch": 5,
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum": [
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         0
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_names": [
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "trial160"
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     ],
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "quorum_age": 5,
2026-02-20T22:39:30.789 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "monmap": {
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "min_mon_release_name": "tentacle",
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_mons": 1
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "osdmap": {
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_osds": 0,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_up_osds": 0,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_up_since": 0,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_in_osds": 0,
2026-02-20T22:39:30.790 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "osd_in_since": 0,
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_remapped_pgs": 0
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "pgmap": {
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "pgs_by_state": [],
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pgs": 0,
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_pools": 0,
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_objects": 0,
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "data_bytes": 0,
2026-02-20T22:39:30.791 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_used": 0,
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_avail": 0,
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "bytes_total": 0
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "fsmap": {
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "btime": "2026-02-20T22:39:24:775392+0000",
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "by_rank": [],
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "up:standby": 0
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "mgrmap": {
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "available": true,
2026-02-20T22:39:30.792 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "num_standbys": 0,
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modules": [
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "iostat",
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout             "nfs"
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         ],
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "servicemap": {
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "epoch": 1,
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "modified": "2026-02-20T22:39:24.775772+0000",
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout         "services": {}
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     },
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "progress_events": {}
2026-02-20T22:39:30.793 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:30.794 INFO:teuthology.orchestra.run.trial160.stdout:mgr is available
2026-02-20T22:39:31.010 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:31.010 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [global]
2026-02-20T22:39:31.010 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	fsid = fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:31.010 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_cluster_log_file_level = debug
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_host = [v2:10.20.193.160:3300,v1:10.20.193.160:6789]
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_osd_allow_pg_remap = true
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_osd_allow_primary_affinity = true
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mon_warn_on_no_sortbitwise = false
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_crush_chooseleaf_type = 0
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [mgr]
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	mgr/telemetry/nag = false
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout [osd]
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_map_max_advance = 10
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 	osd_sloppy_crc = true
2026-02-20T22:39:31.011 INFO:teuthology.orchestra.run.trial160.stdout:Enabling cephadm module...
2026-02-20T22:39:31.057 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:30 trial160 ceph-mon[35528]: mgrmap e3: trial160.pdmeos(active, since 1.00566s)
2026-02-20T22:39:31.057 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:30 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2734110708' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch
2026-02-20T22:39:32.179 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:31 trial160 ceph-mon[35528]: mgrmap e4: trial160.pdmeos(active, since 2s)
2026-02-20T22:39:32.179 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/586194107' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch
2026-02-20T22:39:32.179 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2955453821' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch
2026-02-20T22:39:32.201 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "epoch": 5,
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "available": true,
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "active_name": "trial160.pdmeos",
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "num_standby": 0
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for the mgr to restart...
2026-02-20T22:39:32.202 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mgr epoch 5...
2026-02-20T22:39:33.336 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:32 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2955453821' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished
2026-02-20T22:39:33.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:32 trial160 ceph-mon[35528]: mgrmap e5: trial160.pdmeos(active, since 3s)
2026-02-20T22:39:33.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:32 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2864418300' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: Active manager daemon trial160.pdmeos restarted
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: Activating manager daemon trial160.pdmeos
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: osdmap e2: 0 total, 0 up, 0 in
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: mgrmap e6: trial160.pdmeos(active, starting, since 0.00260118s)
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:39:35.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:39:35.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:34 trial160 ceph-mon[35528]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "mgrmap_epoch": 7,
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "initialized": true
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:mgr epoch 5 is available
2026-02-20T22:39:35.723 INFO:teuthology.orchestra.run.trial160.stdout:Verifying orchestrator module is enabled...
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: Found migration_current of "None". Setting to last migration.
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:36.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: mgrmap e7: trial160.pdmeos(active, since 1.00364s)
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:35] ENGINE Bus STARTING
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:35] ENGINE Serving on http://10.20.193.160:8765
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:35] ENGINE Serving on https://10.20.193.160:7150
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:35] ENGINE Bus STARTED
2026-02-20T22:39:36.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:36.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:35] ENGINE Client ('10.20.193.160', 57296) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)')
2026-02-20T22:39:36.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:36 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3630129580' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch
2026-02-20T22:39:36.730 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on)
2026-02-20T22:39:36.730 INFO:teuthology.orchestra.run.trial160.stdout:Setting orchestrator backend to cephadm...
2026-02-20T22:39:37.220 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout value unchanged
2026-02-20T22:39:37.220 INFO:teuthology.orchestra.run.trial160.stdout:Generating ssh key...
2026-02-20T22:39:37.695 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKT4LlRiyAx6GaDLMrywkid4YMlNdxOKUaFYlAmkN0s6 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:37.695 INFO:teuthology.orchestra.run.trial160.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub
2026-02-20T22:39:37.695 INFO:teuthology.orchestra.run.trial160.stdout:Adding key to root@localhost authorized_keys...
2026-02-20T22:39:37.696 INFO:teuthology.orchestra.run.trial160.stdout:Adding host trial160...
2026-02-20T22:39:37.740 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3630129580' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: mgrmap e8: trial160.pdmeos(active, since 2s)
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:37.741 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:37 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:38.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:38 trial160 ceph-mon[35528]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:38.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:38 trial160 ceph-mon[35528]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:38.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:38 trial160 ceph-mon[35528]: Generating ssh key...
2026-02-20T22:39:38.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:38 trial160 ceph-mon[35528]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:38.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:38 trial160 ceph-mon[35528]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial160", "addr": "10.20.193.160", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:39.495 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Added host 'trial160' with addr '10.20.193.160'
2026-02-20T22:39:39.495 INFO:teuthology.orchestra.run.trial160.stdout:Deploying mon service with default placement...
2026-02-20T22:39:39.727 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:39 trial160 ceph-mon[35528]: Deploying cephadm binary to trial160
2026-02-20T22:39:39.727 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:39 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:39.727 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:39 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:39.739 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled mon update...
2026-02-20T22:39:39.739 INFO:teuthology.orchestra.run.trial160.stdout:Deploying mgr service with default placement...
2026-02-20T22:39:39.991 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled mgr update...
2026-02-20T22:39:39.991 INFO:teuthology.orchestra.run.trial160.stdout:Deploying crash service with default placement...
2026-02-20T22:39:40.236 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled crash update...
2026-02-20T22:39:40.237 INFO:teuthology.orchestra.run.trial160.stdout:Deploying ceph-exporter service with default placement...
2026-02-20T22:39:40.605 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update...
2026-02-20T22:39:40.605 INFO:teuthology.orchestra.run.trial160.stdout:Deploying prometheus service with default placement...
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: Added host trial160
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: Saving service mon spec with placement count:5
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.836 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:40 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:40.859 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled prometheus update...
2026-02-20T22:39:40.859 INFO:teuthology.orchestra.run.trial160.stdout:Deploying grafana service with default placement...
2026-02-20T22:39:41.103 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled grafana update...
2026-02-20T22:39:41.103 INFO:teuthology.orchestra.run.trial160.stdout:Deploying node-exporter service with default placement...
2026-02-20T22:39:41.335 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update...
2026-02-20T22:39:41.335 INFO:teuthology.orchestra.run.trial160.stdout:Deploying alertmanager service with default placement...
2026-02-20T22:39:41.585 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update...
2026-02-20T22:39:41.862 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: Saving service mgr spec with placement count:2
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: Saving service crash spec with placement *
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: Saving service ceph-exporter spec with placement *
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: Saving service prometheus spec with placement count:1
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:41.863 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:41.864 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='mgr.14118 10.20.193.160:0/2347689461' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:41.864 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:41 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2117548792' entity='client.admin'
2026-02-20T22:39:42.046 INFO:teuthology.orchestra.run.trial160.stdout:Enabling the dashboard module...
2026-02-20T22:39:43.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:43.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: Saving service grafana spec with placement count:1
2026-02-20T22:39:43.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:43.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: Saving service node-exporter spec with placement *
2026-02-20T22:39:43.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:43.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: Saving service alertmanager spec with placement count:1
2026-02-20T22:39:43.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/9085785' entity='client.admin'
2026-02-20T22:39:43.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:42 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/96322945' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch
2026-02-20T22:39:43.314 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "epoch": 9,
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "available": true,
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "active_name": "trial160.pdmeos",
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "num_standby": 0
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for the mgr to restart...
2026-02-20T22:39:43.315 INFO:teuthology.orchestra.run.trial160.stdout:Waiting for mgr epoch 9...
2026-02-20T22:39:44.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:44 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/96322945' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished
2026-02-20T22:39:44.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:44 trial160 ceph-mon[35528]: mgrmap e9: trial160.pdmeos(active, since 8s)
2026-02-20T22:39:44.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:44 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1909697586' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch
2026-02-20T22:39:46.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: Active manager daemon trial160.pdmeos restarted
2026-02-20T22:39:46.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: Activating manager daemon trial160.pdmeos
2026-02-20T22:39:46.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: osdmap e3: 0 total, 0 up, 0 in
2026-02-20T22:39:46.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: mgrmap e10: trial160.pdmeos(active, starting, since 0.0026606s)
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:39:46.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:45 trial160 ceph-mon[35528]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:39:47.050 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:46 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:39:47.050 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:46 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:39:47.051 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:46 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:39:47.051 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:46 trial160 ceph-mon[35528]: mgrmap e11: trial160.pdmeos(active, since 1.00493s)
2026-02-20T22:39:47.051 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:46 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "mgrmap_epoch": 11,
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout     "initialized": true
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout }
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:mgr epoch 9 is available
2026-02-20T22:39:47.053 INFO:teuthology.orchestra.run.trial160.stdout:Using certmgr to generate dashboard self-signed certificate...
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:46] ENGINE Bus STARTING
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:46] ENGINE Serving on https://10.20.193.160:7150
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:46] ENGINE Client ('10.20.193.160', 44834) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)')
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:46] ENGINE Serving on http://10.20.193.160:8765
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: [20/Feb/2026:22:39:46] ENGINE Bus STARTED
2026-02-20T22:39:48.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch
2026-02-20T22:39:48.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch
2026-02-20T22:39:48.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:47 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:48.313 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUYK5IyyKpM5dHSY51BYoGoZOc5QYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI2MDIyMDIyMzk0OFoXDTI5\nMDIxOTIyMzk0OFowGDEWMBQGA1UEAwwNMTAuMjAuMTkzLjE2MDCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAKEskWfo5CROcJDLD9+WYj2J4u5MIcObzHMz\nNWuLKY1HZe8aOLCwSyIVgfjIumLf6a7ns6WDK2BYLdMSI7CRYACo47pFccWtwNpX\nGvFvwIfJc4XUuFszUxNEVZ1AR4D9ISsGaKFvozXzRFgQTDmMa7zBtlwcNA40DYU2\nEsdAcC3hBxX/8UA+kmCT/5WQ8R3h3y+d59R6pUk1rcpAVW3oFd6SF79Qs6neM1of\nAyhc+TFtzjLSPnz6lFXO7WZyHhJMk1YVPfUwdmuPQg8v2HyluV4NRloqojbhKKCU\n/xt8xmX1079ZQ9+4qfCrJFeKDNjZd1hjce4G9x2YNz4WHiw4Nxjtc1I3LCL60hJk\nm0SmO8A9pl7mqkzhsJNMqSGF6+aEfXrxcV3cBkC6Aru76qtPdtf4TqM4m82jKq05\nhvOtVwHn6t4bs/QuU0kUxBYENvVYtM1ZD+VI72vosVN/NyqayX/nRuXUUdROb5J7\nAh7XOOIp8gW+v7gVQlogAWVUYdt36dyCAlybFUUkmR5CszAFKKlPrPk9H7m+B9QL\nRVXiOwJ0I1I0U3vG2+wZaRUJ22EzfhTaGmTjKja3X6xLcU8NuXBcv0Dm/pnFUTCj\neI1CrscZSThOlZmv8Jh+rMrkSngj3CqQkLsUJXsdMBfKvcsN4CHlZNxMWSIj/bz1\nvTvDF/0vAgMBAAGjPjA8MCwGA1UdEQQlMCOCCHRyaWFsMTYwghFkYXNoYm9hcmRf\nc2VydmVyc4cEChTBoDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQCQ\nQFMttxyRF7cpzpWj02X1C4dR4PEoXe07NiuFAtNU/BA89wTbj5f370mmfGyMBEpK\n6mbHmq8ZFmWuk445NO5Slv/9EwgYLRJLyN+c+OqTNWHEpqXMQaUxnzUJpn+xYc5f\n/QPZHzY8YUUfmiOjlwpZTGKExnRK9XpfVQ4RDZvmzls1FJqYxdikANY/bleg0zD5\nTm53Q7q70ga0v8Rk5HTazDTT2V56s9wma3TAYXVlNceYqreG9/JyxTYAvcozyxF+\n4x99ht16TgzdxufBrgXfktKwexctG4/CCVhOsrGZbupUUimh/cqNS6B4ubg46moE\ntypDsSFV4dfv7TIFWI/Rsw09vBrJINaiYdZ808DRc1s67uPK+VkV7qiazJ7ojTNK\nUHg5i7UB3k1dCnffJ+U1Fj7OZQKM9+aqa8gQTdKeXeTCRDRRJ+PefKvy60sDXPak\nR+C/tN394S8Dz9Pqz5aQB6O0bauOCAalK+QgNii/hIZ3pBppo+GOGeBzckZcSujj\nglgeEYDRsu/UJKKQn5hVuLTw4qoW/kDSV5snxijj51NWbH4saCPXl8y/TA2FLT+Q\nDhQpjUrIWOUOL18UibYyXW1plAGKoxflUaUp+en9cXGSxQH6mbT0JwfPBXGiVOjK\nyxumJg4O1bhJLifTzIFPVFEbM8RcH2IhrwgdE4hXKw==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAoSyRZ+jkJE5wkMsP35ZiPYni7kwhw5vMczM1a4spjUdl7xo4\nsLBLIhWB+Mi6Yt/pruezpYMrYFgt0xIjsJFgAKjjukVxxa3A2lca8W/Ah8lzhdS4\nWzNTE0RVnUBHgP0hKwZooW+jNfNEWBBMOYxrvMG2XBw0DjQNhTYSx0BwLeEHFf/x\nQD6SYJP/lZDxHeHfL53n1HqlSTWtykBVbegV3pIXv1Czqd4zWh8DKFz5MW3OMtI+\nfPqUVc7tZnIeEkyTVhU99TB2a49CDy/YfKW5Xg1GWiqiNuEooJT/G3zGZfXTv1lD\n37ip8KskV4oM2Nl3WGNx7gb3HZg3PhYeLDg3GO1zUjcsIvrSEmSbRKY7wD2mXuaq\nTOGwk0ypIYXr5oR9evFxXdwGQLoCu7vqq0921/hOozibzaMqrTmG861XAefq3huz\n9C5TSRTEFgQ29Vi0zVkP5Ujva+ixU383KprJf+dG5dRR1E5vknsCHtc44inyBb6/\nuBVCWiABZVRh23fp3IICXJsVRSSZHkKzMAUoqU+s+T0fub4H1AtFVeI7AnQjUjRT\ne8bb7BlpFQnbYTN+FNoaZOMqNrdfrEtxTw25cFy/QOb+mcVRMKN4jUKuxxlJOE6V\nma/wmH6syuRKeCPcKpCQuxQlex0wF8q9yw3gIeVk3ExZIiP9vPW9O8MX/S8CAwEA\nAQKCAgA3hLGluHx14Ss3RSt3mkbG6etoqILs7Yr0l8wr+mjayKBdR4wFmQSYfg4i\nU4qYqobFf/zK468aZeWLEnGGzdzi/UpMjMVPEbYfv+eBCTypyYPossi6aaiRN3XC\nUDURQqCdQHzkNx9iYyTErQCExlEKWTPm8LQhvF5355N6XUEHHL+jHWHN/n/Pkst0\nitPzti16KouEDqdG25280IJc9Jgr7ekONs9ckTThM1OUIMqZ8zgXG4L47BwSns/5\n/Q9MwGH0QibwgbwY+uUHdDS/YxrCQhJc1ti/zLmA7zIdFtMEpjRFcotF0kd/poRD\nP0ODow1QPhkto+qVoCxPT7EfDFI10QP2g4Ti7MbbRUnPNGUl7aDXBwaz5RT4D8zL\nCAVMi8ir52NqobFyuNfdacwx5jR6FXUUORFQ3dRU53ogbOIi732ygrYlZV9mz9ZX\np4F9B70ZZWK9/UX7s4AAHqnm/+8e7VFROBGQLYbsSZjTtuemWJLKqSa9l7JG1Jqy\nFy9EfrU9pyAZvlKn/SO7OV8kz3UuDQ/CzHAonU8uCC7H03BvWZ8CluTuCOA28iqi\n0MYL4Dyjf5B9GGU04EhZPNAEMbW2MRNpwNCY6BlOOPTC7AcVy8iIXhbo2zGqniws\nLYNbXAcyYj4o1u2RvFF+v704x3a5ijJ4Lhv7DUAHckSYN7L8oQKCAQEAz5tDrbP9\nn3woYfx6W5wA3LVL2Z0oRY7W39ne4PImyjHNpMLTK0liqBSiOla9yWnM3rCyOuI9\np/ed0QSCiAuT4TQQOS4stJDl7CHiUPJv7jmIN0uURhnuBYrtDAnNA/YT2hlG07mw\ntgDY8Y/1/2rf0mKfLmvu2EOOxj/4pVaSsccRl9ow+h4kMvu2B6c87e/hozIr3+/4\nmEl8IIBkICX8OZk8k8CEKG11UftatroULXovG1Oblr95PBirnPIaGTMs5H9Ac3EM\nTwTnJcdap7B1guVU5lpidYJOGZDfRD7FcRUKyV8kNqdFxHnRBFjHd9fvc4ZJsWd4\nL/R4+ldeeCjBUQKCAQEAxr59j63Y1qM9qeJQbgX/Gb2YRkmKZmSm0EwAgwUa7EHQ\nkPHg2ayXAqvQhqDqYL/fHPAOynkvpYBBIt7Ri/XfKHdM41H9AXp9YqiUrnU4ZGnF\nW98QLiRCcAqQmrsVyUmTK1+xPnTS1MKTHHTDowBsgR6fNSY5lv2nhIM9VFH4US2p\n4I4EKS03gz78gtMWv4S9n7C5sr9FzBSl67CiZgKH/cxjDsUlwIoRSCov54EAv2JO\nRluArbjNLo9SuXuCi+FWBjsji8hRy2nYxMV2roVSuRNQ6QcTO+Yq6sDJ0wNJBEiF\nG26nd0f5HB6xuDs1fzLtEVUDtCDWG8gqiweUDkk2fwKCAQBBjp79gCPrs9IwtwCQ\nYx/mkh1kl3r7FBMIVw1JbU4QdU+h4p+6QI810mY0E+YUbyNxgsyVSZH8s3WTZmAt\nw7Ir4DvrQoh9cHAac8y6kQQFbwdzppso09FU0LzwGzwxLwivy2GRREboAjZ/eV1O\nDZqmHdx0VULz6clxEwxacY8bMmh80jq7EL+PleqyWf4kiKZw2Uv/iBCxTiszPDfh\n6vzN13TIl3dkKMRuYfgQUYxDnBy5/VnZbZdEAepXQiA5qu5XLVPV4HHplfvD80LO\nRyukbPS1VP9YahBthFDshQHmJqXkVvXhiErAs1Lsk3hRgM+tZE9rBOhwBDwbmXFM\ncloxAoIBAFg0ZsmDoH7yCAYDuSp7YtJrwM9BCDmVZit/76I2e9JrC0Oo9d/lp0e6\nNVvqwtqZjU3shMP7sNfE6iOtvFZfkuEEwY0J8D3dvRQyOXqL77Ew+DRj9WxlD09m\ngPMLIQoqqzROzf0oIAHpMR0p/UG0iHNtwb727gxDGzvGe0uW7ZwppgFRRit4M/D9\n3Ve6k5gAM6BR8LiSB0ohx9molQqtCOns8fG2MaEYJb9fac9XA5vs50x2g8+V+wo0\n9cP4OuIV6fXt/UKmbxbWLSx3bsrZxPL6zaf+Sd0Fs/poAaqzBI7wGJMqMwzPVpHS\nVZGsRXvkgC53CqPBBwVhevAlNuJ4P8UCggEAFoAzJbt3zHcxBiSmQSIO75S+zygR\nwDdRt/l6TVeCB69msFJ+yEr9a0kCYkVuQCxoVPOATaGn0KWM3F3tsh/rzYWhMvzf\nHajGx/iMkQUesl7G9CDVnGerPP2+a0O5YhGBcRV+uqrjeY/ogavd8bUu/weKS+2i\nhoF0V09RhspiG4zyM5XNMO0LXkqLxhg4CfwAN+rEO9L0Ex4cCKg/50u+f4UeHF6r\nqExruXP82uZuqJgcvlmAoiIaG/UQkZbL6LlCxjsnIJRZLt5bekfAalPI5pNiMMo4\nIdd3OBqMzTPFLRsqfimjgE5acaJ8lqJokKnF6NdsH0q2szMu/nMMGGLk5g==\n-----END RSA PRIVATE KEY-----\n"}
2026-02-20T22:39:48.553 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout SSL certificate updated
2026-02-20T22:39:48.794 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout SSL certificate key updated
2026-02-20T22:39:48.794 INFO:teuthology.orchestra.run.trial160.stdout:Creating initial admin user...
2026-02-20T22:39:49.086 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:48 trial160 ceph-mon[35528]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:49.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:48 trial160 ceph-mon[35528]: mgrmap e12: trial160.pdmeos(active, since 2s)
2026-02-20T22:39:49.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:48 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:49.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:48 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:49.240 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$HYETzg9N6hFWt0Qcy56/pOx0o7eNgi4qF/fKAV1PJFCKkmFA8jEAm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1771627189, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true}
2026-02-20T22:39:49.240 INFO:teuthology.orchestra.run.trial160.stdout:Fetching dashboard port number...
2026-02-20T22:39:49.468 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stdout 8443
2026-02-20T22:39:49.477 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service
2026-02-20T22:39:49.477 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout disabled
2026-02-20T22:39:49.484 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service
2026-02-20T22:39:49.484 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:49.484 INFO:teuthology.orchestra.run.trial160.stdout:firewalld.service is not enabled
2026-02-20T22:39:49.484 INFO:teuthology.orchestra.run.trial160.stdout:Not possible to open ports <[8443]>. firewalld.service is not available
2026-02-20T22:39:49.485 INFO:teuthology.orchestra.run.trial160.stdout:Ceph Dashboard is now available at:
2026-02-20T22:39:49.485 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.485 INFO:teuthology.orchestra.run.trial160.stdout:	     URL: https://trial160:8443/
2026-02-20T22:39:49.485 INFO:teuthology.orchestra.run.trial160.stdout:	    User: admin
2026-02-20T22:39:49.486 INFO:teuthology.orchestra.run.trial160.stdout:	Password: isuy87ax3t
2026-02-20T22:39:49.486 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.486 INFO:teuthology.orchestra.run.trial160.stdout:Saving cluster configuration to /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config directory
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:	sudo /home/ubuntu/cephtest/cephadm shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:Or, if you are only running a single cluster on this host:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:	sudo /home/ubuntu/cephtest/cephadm shell
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:Please consider enabling telemetry to help improve Ceph:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:	ceph telemetry on
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:For more information see:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:	https://docs.ceph.com/en/latest/mgr/telemetry/
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:49.770 INFO:teuthology.orchestra.run.trial160.stdout:Bootstrap complete.
2026-02-20T22:39:49.779 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout static
2026-02-20T22:39:49.785 INFO:teuthology.orchestra.run.trial160.stdout:Non-zero exit code 3 from systemctl is-active logrotate
2026-02-20T22:39:49.785 INFO:teuthology.orchestra.run.trial160.stdout:systemctl: stdout inactive
2026-02-20T22:39:49.785 INFO:teuthology.orchestra.run.trial160.stdout:Enabling the logrotate.timer service to perform daily log rotation.
2026-02-20T22:39:49.936 INFO:tasks.cephadm:Fetching config...
2026-02-20T22:39:49.936 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:49.936 DEBUG:teuthology.orchestra.run.trial160:> dd if=/etc/ceph/ceph.conf of=/dev/stdout
2026-02-20T22:39:49.953 INFO:tasks.cephadm:Fetching client.admin keyring...
2026-02-20T22:39:49.954 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:49.954 DEBUG:teuthology.orchestra.run.trial160:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/723589490' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch
2026-02-20T22:39:50.001 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1022438341' entity='client.admin'
2026-02-20T22:39:50.011 INFO:tasks.cephadm:Fetching mon keyring...
2026-02-20T22:39:50.011 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:50.012 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/keyring of=/dev/stdout
2026-02-20T22:39:50.077 INFO:tasks.cephadm:Fetching pub ssh key...
2026-02-20T22:39:50.078 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:39:50.078 DEBUG:teuthology.orchestra.run.trial160:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout
2026-02-20T22:39:50.134 INFO:tasks.cephadm:Installing pub ssh key for root users...
2026-02-20T22:39:50.134 DEBUG:teuthology.orchestra.run.trial160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKT4LlRiyAx6GaDLMrywkid4YMlNdxOKUaFYlAmkN0s6 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys
2026-02-20T22:39:50.207 INFO:teuthology.orchestra.run.trial160.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKT4LlRiyAx6GaDLMrywkid4YMlNdxOKUaFYlAmkN0s6 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:50.217 DEBUG:teuthology.orchestra.run.trial192:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKT4LlRiyAx6GaDLMrywkid4YMlNdxOKUaFYlAmkN0s6 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys
2026-02-20T22:39:50.253 INFO:teuthology.orchestra.run.trial192.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKT4LlRiyAx6GaDLMrywkid4YMlNdxOKUaFYlAmkN0s6 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:39:50.262 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph config set mgr mgr/cephadm/allow_ptrace true
2026-02-20T22:39:50.377 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:39:50.639 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755
2026-02-20T22:39:50.639 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch client-keyring set client.admin '*' --mode 0755
2026-02-20T22:39:50.758 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:39:51.031 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial192
2026-02-20T22:39:51.031 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:39:51.031 DEBUG:teuthology.orchestra.run.trial192:> dd of=/etc/ceph/ceph.conf
2026-02-20T22:39:51.047 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:39:51.047 DEBUG:teuthology.orchestra.run.trial192:> dd of=/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:39:51.105 INFO:tasks.cephadm:Adding host trial192 to orchestrator...
2026-02-20T22:39:51.105 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch host add trial192
2026-02-20T22:39:51.224 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:39:51.655 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1073215267' entity='client.admin'
2026-02-20T22:39:51.655 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd/host:trial160", "name": "osd_memory_target"} : dispatch
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial160", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial160", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: Deploying daemon ceph-exporter.trial160 on trial160
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:51.656 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:51 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial192", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: Deploying cephadm binary to trial192
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:52.818 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:52.819 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.crash.trial160", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch
2026-02-20T22:39:52.819 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.trial160", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished
2026-02-20T22:39:52.819 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:52 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:39:53.062 INFO:teuthology.orchestra.run.trial160.stdout:Added host 'trial192' with addr '10.20.193.192'
2026-02-20T22:39:53.102 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch host ls --format=json
2026-02-20T22:39:53.229 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:39:53.467 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:39:53.467 INFO:teuthology.orchestra.run.trial160.stdout:[{"addr": "10.20.193.160", "hostname": "trial160", "labels": [], "status": ""}, {"addr": "10.20.193.192", "hostname": "trial192", "labels": [], "status": ""}]
2026-02-20T22:39:53.508 INFO:tasks.cephadm:Setting crush tunables to default
2026-02-20T22:39:53.509 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd crush tunables default
2026-02-20T22:39:53.627 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:39:53.881 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: Deploying daemon crash.trial160 on trial160
2026-02-20T22:39:53.881 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: mgrmap e13: trial160.pdmeos(active, since 6s)
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: Deploying daemon node-exporter.trial160 on trial160
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:53.882 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:53 trial160 ceph-mon[35528]: Added host trial192
2026-02-20T22:39:54.607 INFO:teuthology.orchestra.run.trial160.stderr:adjusted tunables profile to default
2026-02-20T22:39:54.647 INFO:tasks.cephadm:Adding mon.trial160 on trial160
2026-02-20T22:39:54.647 INFO:tasks.cephadm:Adding mon.trial192 on trial192
2026-02-20T22:39:54.647 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch apply mon '2;trial160:10.20.193.160=trial160;trial192:10.20.193.192=trial192'
2026-02-20T22:39:54.768 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:54.768 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:55.010 INFO:teuthology.orchestra.run.trial192.stdout:Scheduled mon update...
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3063698909' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:54 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.058 DEBUG:teuthology.orchestra.run.trial192:mon.trial192> sudo journalctl -f -n 0 -u ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial192.service
2026-02-20T22:39:55.060 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:39:55.061 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:39:55.214 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:55.214 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:55.498 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:55.498 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:39:55.498 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: Deploying daemon alertmanager.trial160 on trial160
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3063698909' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: osdmap e4: 0 total, 0 up, 0 in
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='client.14199 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;trial160:10.20.193.160=trial160;trial192:10.20.193.192=trial192", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: Saving service mon spec with placement trial160:10.20.193.160=trial160;trial192:10.20.193.192=trial192;count:2
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/131836968' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:55.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:55 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:56.549 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:39:56.549 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:39:56.670 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:56.671 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:56.936 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:56.936 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:39:56.937 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:39:57.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: Generating cephadm-signed certificates for grafana_cert/grafana_key
2026-02-20T22:39:57.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch
2026-02-20T22:39:57.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:39:57.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:56 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/283220233' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:39:57.974 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:39:57.975 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:39:58.096 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:58.096 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:58.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:57 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch
2026-02-20T22:39:58.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:57 trial160 ceph-mon[35528]: Deploying daemon grafana.trial160 on trial160
2026-02-20T22:39:58.363 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:58.363 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:39:58.363 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:39:59.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:58 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/2214663199' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:39:59.400 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:39:59.400 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:39:59.516 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:59.516 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:39:59.784 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:39:59.785 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:39:59.785 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:00.124 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:39:59 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3712623376' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:00.820 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:00.821 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:00.939 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:00.939 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:01.218 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:01.218 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:01.218 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:01.685 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.685 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.685 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.685 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: Deploying daemon prometheus.trial160 on trial160
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:01.686 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:01 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/584313844' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:02.255 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:02.255 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:02.373 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:02.373 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:02.648 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:02.649 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:02.649 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:03.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:02 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/1562119800' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:03.709 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:03.709 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:03.843 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:03.843 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:04.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:03 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:04.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:03 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:04.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:03 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:04.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:03 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch
2026-02-20T22:40:04.115 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:04.115 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:04.115 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:05.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:04 trial160 ceph-mon[35528]: from='mgr.14164 10.20.193.160:0/3194458288' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished
2026-02-20T22:40:05.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:04 trial160 ceph-mon[35528]: mgrmap e14: trial160.pdmeos(active, since 17s)
2026-02-20T22:40:05.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:04 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/742016601' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:05.154 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:05.154 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:05.271 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:05.272 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:05.543 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:05.543 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:05.543 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:05.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:05 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3201329143' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:06.582 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:06.582 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:06.703 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:06.703 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:06.978 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:06.978 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:06.979 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: Active manager daemon trial160.pdmeos restarted
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: Activating manager daemon trial160.pdmeos
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: osdmap e5: 0 total, 0 up, 0 in
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: mgrmap e15: trial160.pdmeos(active, starting, since 0.00267851s)
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:07.071 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:40:07.072 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:06 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:40:08.017 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:08.017 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:08.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/2344984435' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:08.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:08.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:08.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: mgrmap e16: trial160.pdmeos(active, since 1.00469s)
2026-02-20T22:40:08.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:08.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:07 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:08.134 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:08.135 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /etc/ceph/ceph.conf
2026-02-20T22:40:08.484 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:08.484 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:08.485 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: [20/Feb/2026:22:40:07] ENGINE Bus STARTING
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: [20/Feb/2026:22:40:07] ENGINE Serving on https://10.20.193.160:7150
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: [20/Feb/2026:22:40:07] ENGINE Client ('10.20.193.160', 54144) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)')
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: [20/Feb/2026:22:40:07] ENGINE Serving on http://10.20.193.160:8765
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: [20/Feb/2026:22:40:07] ENGINE Bus STARTED
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd/host:trial160", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:09.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:08 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/2721654996' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:09.524 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:09.525 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:09.644 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:09.933 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:09.934 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:09.934 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:10.337 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: mgrmap e17: trial160.pdmeos(active, since 2s)
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd/host:trial192", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial160:/etc/ceph/ceph.conf
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial192:/etc/ceph/ceph.conf
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial192:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial160:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:40:10.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: Deploying daemon ceph-exporter.trial192 on trial192
2026-02-20T22:40:10.339 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:10 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/1518850732' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:11.048 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:11.049 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:11.229 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:11.520 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:11.521 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:11.521 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.crash.trial192", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.trial192", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished
2026-02-20T22:40:12.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: Deploying daemon crash.trial192 on trial192
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3028852883' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:12.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:11 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:12.560 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:12.561 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:12.682 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:12.952 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:12.952 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":1,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:39:23.811880Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]}
2026-02-20T22:40:12.952 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 1
2026-02-20T22:40:13.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: Deploying daemon node-exporter.trial192 on trial192
2026-02-20T22:40:13.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "mgr.trial192.inkpau", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.trial192.inkpau", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr services"} : dispatch
2026-02-20T22:40:13.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:12 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:14.002 INFO:tasks.cephadm:Waiting for 2 mons in monmap...
2026-02-20T22:40:14.002 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mon dump -f json
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: Deploying daemon mgr.trial192.inkpau on trial192
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/2122776286' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "mon."} : dispatch
2026-02-20T22:40:14.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:13 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:14.128 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial192/config
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: mon.trial160 calling monitor election
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/2872291479' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/crt"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: mon.trial192 calling monitor election
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.588 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: mon.trial160 is new leader, mons trial160,trial192 in quorum (ranks 0,1)
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: monmap epoch 2
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: last_changed 2026-02-20T22:40:14.127470+0000
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: created 2026-02-20T22:39:23.811880+0000
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: min_mon_release 20 (tentacle)
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: election_strategy: 1
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: 0: [v2:10.20.193.160:3300/0,v1:10.20.193.160:6789/0] mon.trial160
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: 1: [v2:10.20.193.192:3300/0,v1:10.20.193.192:6789/0] mon.trial192
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: fsmap
2026-02-20T22:40:19.589 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: osdmap e5: 0 total, 0 up, 0 in
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: mgrmap e17: trial160.pdmeos(active, since 12s)
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: overall HEALTH_OK
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: Standby manager daemon trial192.inkpau started
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/2872291479' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/2872291479' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/key"} : dispatch
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/2872291479' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:19.590 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:19 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: mgrmap e18: trial160.pdmeos(active, since 12s), standbys: trial192.inkpau
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial192.inkpau", "id": "trial192.inkpau"} : dispatch
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: Updating trial160:/etc/ceph/ceph.conf
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: Updating trial192:/etc/ceph/ceph.conf
2026-02-20T22:40:20.370 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "mon."} : dispatch
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.371 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:20.372 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "mgr.trial160.pdmeos", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch
2026-02-20T22:40:20.372 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr services"} : dispatch
2026-02-20T22:40:20.372 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:20 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:20.407 INFO:teuthology.orchestra.run.trial192.stdout:
2026-02-20T22:40:20.407 INFO:teuthology.orchestra.run.trial192.stdout:{"epoch":2,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","modified":"2026-02-20T22:40:14.127470Z","created":"2026-02-20T22:39:23.811880Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"trial160","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:3300","nonce":0},{"type":"v1","addr":"10.20.193.160:6789","nonce":0}]},"addr":"10.20.193.160:6789/0","public_addr":"10.20.193.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"trial192","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:3300","nonce":0},{"type":"v1","addr":"10.20.193.192:6789","nonce":0}]},"addr":"10.20.193.192:6789/0","public_addr":"10.20.193.192:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]}
2026-02-20T22:40:20.407 INFO:teuthology.orchestra.run.trial192.stderr:dumped monmap epoch 2
2026-02-20T22:40:20.471 INFO:tasks.cephadm:Generating final ceph.conf file...
2026-02-20T22:40:20.471 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph config generate-minimal-conf
2026-02-20T22:40:20.587 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:20.811 INFO:teuthology.orchestra.run.trial160.stdout:# minimal ceph.conf for fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:40:20.812 INFO:teuthology.orchestra.run.trial160.stdout:[global]
2026-02-20T22:40:20.812 INFO:teuthology.orchestra.run.trial160.stdout:	fsid = fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:40:20.812 INFO:teuthology.orchestra.run.trial160.stdout:	mon_host = [v2:10.20.193.160:3300/0,v1:10.20.193.160:6789/0] [v2:10.20.193.192:3300/0,v1:10.20.193.192:6789/0]
2026-02-20T22:40:20.850 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring...
2026-02-20T22:40:20.851 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:40:20.851 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/ceph/ceph.conf
2026-02-20T22:40:20.878 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:40:20.878 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:40:20.945 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:40:20.945 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/etc/ceph/ceph.conf
2026-02-20T22:40:20.973 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:40:20.973 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:40:21.037 DEBUG:tasks.cephadm:set 0 configs
2026-02-20T22:40:21.037 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph config dump
2026-02-20T22:40:21.152 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:WHO         MASK  LEVEL     OPTION                                          VALUE                                                                                              RO
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:global            dev       auth_debug                                      true
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:global            basic     container_image                                 quay.ceph.io/ceph-ci/ceph@sha256:9cc281dd1e60310f8109166bc76e656ce4c5c9b7d9bb9abfc9aa033dfa762ae4  *
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:global            dev       debug_asserts_on_shutdown                       true
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:global            basic     log_to_file                                     true
2026-02-20T22:40:21.366 INFO:teuthology.orchestra.run.trial160.stdout:global            basic     log_to_journald                                 false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            basic     log_to_stderr                                   false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_allow_pool_delete                           true
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_clock_drift_allowed                         1.000000
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_cluster_log_to_file                         true
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_max_pg_per_osd                              10000
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_pg_warn_max_object_skew                     0.000000
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_warn_on_crush_straw_calc_version_zero       false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_warn_on_legacy_crush_tunables               false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_warn_on_osd_down_out_interval_zero          false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            dev       mon_warn_on_pool_pg_num_not_power_of_two        false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  mon_warn_on_too_few_osds                        false
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            dev       ms_die_on_bug                                   true
2026-02-20T22:40:21.367 INFO:teuthology.orchestra.run.trial160.stdout:global            dev       ms_die_on_old_message                           true
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  osd_pool_default_erasure_code_profile           plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  osd_pool_default_pg_autoscale_mode              off
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:global            advanced  public_network                                  10.20.192.0/20                                                                                     *
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  auth_allow_insecure_global_id_reclaim           false
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  auth_mon_ticket_ttl                             660.000000
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  auth_service_ticket_ttl                         240.000000
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  debug_mon                                       20/20
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  debug_ms                                        1/1
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  debug_paxos                                     20/20
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_data_avail_warn                             5
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_mgr_mkfs_grace                              240
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               dev       mon_osd_prime_pg_temp                           true
2026-02-20T22:40:21.368 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_osd_reporter_subtree_level                  osd
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_reweight_min_bytes_per_osd                  10
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_reweight_min_pgs_per_osd                    4
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_warn_on_insecure_global_id_reclaim          false
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mon               advanced  mon_warn_on_insecure_global_id_reclaim_allowed  false
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  debug_mgr                                       20/20
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  debug_ms                                        1/1
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/cephadm/allow_ptrace                        true                                                                                               *
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/cephadm/container_init                      True                                                                                               *
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/cephadm/migration_current                   7                                                                                                  *
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/dashboard/GRAFANA_API_SSL_VERIFY            false                                                                                              *
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/dashboard/ssl_server_port                   8443                                                                                               *
2026-02-20T22:40:21.369 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mgr/orchestrator/orchestrator                   cephadm
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mon_reweight_min_bytes_per_osd                  10
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:mgr               advanced  mon_reweight_min_pgs_per_osd                    4
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       bdev_debug_aio                                  true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  debug_ms                                        1/1
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  debug_osd                                       20/20
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_misdirected_ops                       true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_op_order                              true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_pg_log_writeout                       true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_shutdown                              true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_verify_cached_snaps                   true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_verify_missing_on_start               true
2026-02-20T22:40:21.370 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_debug_verify_stray_on_activate              true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_deep_scrub_update_digest_min_age            30
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_mclock_profile                              high_recovery_ops
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               dev       osd_mclock_skip_benchmark                       true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_memory_target_autotune                      true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_op_queue                                    debug_random                                                                                       *
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_op_queue_cut_off                            debug_random                                                                                       *
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_recover_clone_overlap                       true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_recovery_max_chunk                          1048576
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_scrub_load_threshold                        5.000000
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_scrub_max_interval                          600.000000
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:osd               advanced  osd_shutdown_pgref_assert                       true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:client.rgw        advanced  rgw_cache_enabled                               true
2026-02-20T22:40:21.371 INFO:teuthology.orchestra.run.trial160.stdout:client.rgw        advanced  rgw_enable_ops_log                              true
2026-02-20T22:40:21.372 INFO:teuthology.orchestra.run.trial160.stdout:client.rgw        advanced  rgw_enable_usage_log                            true
2026-02-20T22:40:21.372 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:21 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.406 INFO:tasks.cephadm:Deploying OSDs...
2026-02-20T22:40:21.406 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:40:21.406 DEBUG:teuthology.orchestra.run.trial160:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:40:21.424 DEBUG:teuthology.misc:devs=['/dev/nvme3n1', '/dev/nvme4n1', '/dev/nvme5n1', '/dev/nvme6n1']
2026-02-20T22:40:21.424 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/nvme3n1
2026-02-20T22:40:21.432 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring mon.trial160 (unknown last config time)...
2026-02-20T22:40:21.432 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring daemon mon.trial160 on trial160
2026-02-20T22:40:21.432 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring mgr.trial160.pdmeos (unknown last config time)...
2026-02-20T22:40:21.432 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring daemon mgr.trial160.pdmeos on trial160
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3606530044' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring ceph-exporter.trial160 (monmap changed)...
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial160", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring daemon ceph-exporter.trial160 on trial160
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring crash.trial160 (monmap changed)...
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.crash.trial160", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch
2026-02-20T22:40:21.433 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:21.434 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: Reconfiguring daemon crash.trial160 on trial160
2026-02-20T22:40:21.434 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/128555554' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:21.434 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.434 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:21 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:21.451 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/nvme3n1
2026-02-20T22:40:21.451 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:21.451 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1325        Links: 1     Device type: 103,6
2026-02-20T22:40:21.451 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:21.452 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:21.452 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:51.495345246 +0000
2026-02-20T22:40:21.452 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:07.503740333 +0000
2026-02-20T22:40:21.452 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:07.503740333 +0000
2026-02-20T22:40:21.452 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:06.570774576 +0000
2026-02-20T22:40:21.452 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1
2026-02-20T22:40:21.516 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:40:21.516 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:40:21.517 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000195296 s, 2.6 MB/s
2026-02-20T22:40:21.518 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1
2026-02-20T22:40:21.576 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/nvme4n1
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/nvme4n1
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1337        Links: 1     Device type: 103,8
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:51.533343857 +0000
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:07.903725653 +0000
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:07.903725653 +0000
2026-02-20T22:40:21.633 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:06.709769475 +0000
2026-02-20T22:40:21.634 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1
2026-02-20T22:40:21.697 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:40:21.697 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:40:21.697 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000185527 s, 2.8 MB/s
2026-02-20T22:40:21.698 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1
2026-02-20T22:40:21.754 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/nvme5n1
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/nvme5n1
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1349        Links: 1     Device type: 103,a
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:51.571342469 +0000
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:08.303710972 +0000
2026-02-20T22:40:21.809 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:08.303710972 +0000
2026-02-20T22:40:21.810 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:06.838764740 +0000
2026-02-20T22:40:21.810 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/nvme5n1 of=/dev/null count=1
2026-02-20T22:40:21.871 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:40:21.871 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:40:21.871 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000188032 s, 2.7 MB/s
2026-02-20T22:40:21.872 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/nvme5n1
2026-02-20T22:40:21.932 DEBUG:teuthology.orchestra.run.trial160:> stat /dev/nvme6n1
2026-02-20T22:40:21.991 INFO:teuthology.orchestra.run.trial160.stdout:  File: /dev/nvme6n1
2026-02-20T22:40:21.991 INFO:teuthology.orchestra.run.trial160.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Device: 6h/6d	Inode: 1361        Links: 1     Device type: 103,c
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Access: 2026-02-20 22:39:51.608341117 +0000
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Modify: 2026-02-20 22:39:08.700915686 +0000
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout:Change: 2026-02-20 22:39:08.700915686 +0000
2026-02-20T22:40:21.992 INFO:teuthology.orchestra.run.trial160.stdout: Birth: 2026-02-20 22:39:06.971759859 +0000
2026-02-20T22:40:21.992 DEBUG:teuthology.orchestra.run.trial160:> sudo dd if=/dev/nvme6n1 of=/dev/null count=1
2026-02-20T22:40:22.055 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records in
2026-02-20T22:40:22.056 INFO:teuthology.orchestra.run.trial160.stderr:1+0 records out
2026-02-20T22:40:22.056 INFO:teuthology.orchestra.run.trial160.stderr:512 bytes copied, 0.000175689 s, 2.9 MB/s
2026-02-20T22:40:22.057 DEBUG:teuthology.orchestra.run.trial160:> ! mount | grep -v devtmpfs | grep -q /dev/nvme6n1
2026-02-20T22:40:22.115 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:40:22.116 DEBUG:teuthology.orchestra.run.trial192:> dd if=/scratch_devs of=/dev/stdout
2026-02-20T22:40:22.134 DEBUG:teuthology.misc:devs=['/dev/nvme3n1', '/dev/nvme4n1', '/dev/nvme5n1', '/dev/nvme6n1']
2026-02-20T22:40:22.135 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/nvme3n1
2026-02-20T22:40:22.191 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/nvme3n1
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1325        Links: 1     Device type: 103,6
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:40:08.879997822 +0000
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:10.711243969 +0000
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:10.711243969 +0000
2026-02-20T22:40:22.192 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:09.775218760 +0000
2026-02-20T22:40:22.193 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1
2026-02-20T22:40:22.254 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:40:22.254 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:40:22.254 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000186367 s, 2.7 MB/s
2026-02-20T22:40:22.256 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1
2026-02-20T22:40:22.313 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/nvme4n1
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/nvme4n1
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1337        Links: 1     Device type: 103,8
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:22.371 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:40:08.917997391 +0000
2026-02-20T22:40:22.372 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:11.113265739 +0000
2026-02-20T22:40:22.372 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:11.113265739 +0000
2026-02-20T22:40:22.372 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:09.911217050 +0000
2026-02-20T22:40:22.372 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: Reconfiguring alertmanager.trial160 deps ['mgr.trial160.pdmeos', 'secure_monitoring_stack:False'] -> ['alertmanager.trial160', 'mgr.trial160.pdmeos', 'mgr.trial192.inkpau', 'secure_monitoring_stack:False'] (diff {'mgr.trial192.inkpau', 'alertmanager.trial160'})
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: Reconfiguring daemon alertmanager.trial160 on trial160
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4007996899' entity='client.admin' cmd={"prefix": "config dump"} : dispatch
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:22.375 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: Reconfiguring grafana.trial160 deps ['secure_monitoring_stack:False'] -> ['prometheus.trial160', 'secure_monitoring_stack:False'] (diff {'prometheus.trial160'})
2026-02-20T22:40:22.376 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:22.376 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch
2026-02-20T22:40:22.376 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch
2026-02-20T22:40:22.376 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:22 trial160 ceph-mon[35528]: Reconfiguring daemon grafana.trial160 on trial160
2026-02-20T22:40:22.434 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:40:22.434 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:40:22.434 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000190314 s, 2.7 MB/s
2026-02-20T22:40:22.435 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1
2026-02-20T22:40:22.493 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/nvme5n1
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/nvme5n1
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1349        Links: 1     Device type: 103,a
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:40:08.955996960 +0000
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:11.513287401 +0000
2026-02-20T22:40:22.549 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:11.513287401 +0000
2026-02-20T22:40:22.550 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:10.044215378 +0000
2026-02-20T22:40:22.550 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/nvme5n1 of=/dev/null count=1
2026-02-20T22:40:22.610 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: Reconfiguring alertmanager.trial160 deps ['mgr.trial160.pdmeos', 'secure_monitoring_stack:False'] -> ['alertmanager.trial160', 'mgr.trial160.pdmeos', 'mgr.trial192.inkpau', 'secure_monitoring_stack:False'] (diff {'mgr.trial192.inkpau', 'alertmanager.trial160'})
2026-02-20T22:40:22.610 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: Reconfiguring daemon alertmanager.trial160 on trial160
2026-02-20T22:40:22.610 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4007996899' entity='client.admin' cmd={"prefix": "config dump"} : dispatch
2026-02-20T22:40:22.610 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: Reconfiguring grafana.trial160 deps ['secure_monitoring_stack:False'] -> ['prometheus.trial160', 'secure_monitoring_stack:False'] (diff {'prometheus.trial160'})
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch
2026-02-20T22:40:22.611 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:22 trial192 ceph-mon[41689]: Reconfiguring daemon grafana.trial160 on trial160
2026-02-20T22:40:22.612 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:40:22.612 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:40:22.612 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000163104 s, 3.1 MB/s
2026-02-20T22:40:22.613 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/nvme5n1
2026-02-20T22:40:22.670 DEBUG:teuthology.orchestra.run.trial192:> stat /dev/nvme6n1
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:  File: /dev/nvme6n1
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:  Size: 0         	Blocks: 0          IO Block: 4096   block special file
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:Device: 6h/6d	Inode: 1361        Links: 1     Device type: 103,c
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:Access: (0660/brw-rw----)  Uid: (    0/    root)   Gid: (    6/    disk)
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:Context: system_u:object_r:fixed_disk_device_t:s0
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:Access: 2026-02-20 22:40:08.993996530 +0000
2026-02-20T22:40:22.727 INFO:teuthology.orchestra.run.trial192.stdout:Modify: 2026-02-20 22:39:11.915309174 +0000
2026-02-20T22:40:22.728 INFO:teuthology.orchestra.run.trial192.stdout:Change: 2026-02-20 22:39:11.915309174 +0000
2026-02-20T22:40:22.728 INFO:teuthology.orchestra.run.trial192.stdout: Birth: 2026-02-20 22:39:10.182215321 +0000
2026-02-20T22:40:22.728 DEBUG:teuthology.orchestra.run.trial192:> sudo dd if=/dev/nvme6n1 of=/dev/null count=1
2026-02-20T22:40:22.790 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records in
2026-02-20T22:40:22.791 INFO:teuthology.orchestra.run.trial192.stderr:1+0 records out
2026-02-20T22:40:22.791 INFO:teuthology.orchestra.run.trial192.stderr:512 bytes copied, 0.000162203 s, 3.2 MB/s
2026-02-20T22:40:22.792 DEBUG:teuthology.orchestra.run.trial192:> ! mount | grep -v devtmpfs | grep -q /dev/nvme6n1
2026-02-20T22:40:22.849 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch apply osd --all-available-devices
2026-02-20T22:40:23.004 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial192/config
2026-02-20T22:40:23.231 INFO:teuthology.orchestra.run.trial192.stdout:Scheduled osd.all-available-devices update...
2026-02-20T22:40:23.289 INFO:tasks.cephadm:Waiting for 8 OSDs to come up...
2026-02-20T22:40:23.290 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:23.410 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:23.638 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:23.678 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0}
2026-02-20T22:40:23.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: Reconfiguring prometheus.trial160 deps ['8765', '9283', 'alertmanager', 'ceph-exporter.trial160', 'mgr.trial160.pdmeos', 'node-exporter', 'secure_monitoring_stack:False'] -> ['8765', '9283', 'alertmanager', 'ceph-exporter.trial160', 'ceph-exporter.trial192', 'mgr.trial160.pdmeos', 'node-exporter', 'secure_monitoring_stack:False'] (diff {'ceph-exporter.trial192'})
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: Reconfiguring daemon prometheus.trial160 on trial160
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:23.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:23 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1013184919' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:23.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: Reconfiguring prometheus.trial160 deps ['8765', '9283', 'alertmanager', 'ceph-exporter.trial160', 'mgr.trial160.pdmeos', 'node-exporter', 'secure_monitoring_stack:False'] -> ['8765', '9283', 'alertmanager', 'ceph-exporter.trial160', 'ceph-exporter.trial192', 'mgr.trial160.pdmeos', 'node-exporter', 'secure_monitoring_stack:False'] (diff {'ceph-exporter.trial192'})
2026-02-20T22:40:23.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: Reconfiguring daemon prometheus.trial160 on trial160
2026-02-20T22:40:23.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:23.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.trial192", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch
2026-02-20T22:40:23.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:23.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:23 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1013184919' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:24.680 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:24.800 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Marking host: trial160 for OSDSpec preview refresh.
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Marking host: trial192 for OSDSpec preview refresh.
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Saving service osd.all-available-devices spec with placement *
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Reconfiguring ceph-exporter.trial192 (monmap changed)...
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Reconfiguring daemon ceph-exporter.trial192 on trial192
2026-02-20T22:40:25.030 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Reconfiguring crash.trial192 (monmap changed)...
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.crash.trial192", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: Reconfiguring daemon crash.trial192 on trial192
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "mgr.trial192.inkpau", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr services"} : dispatch
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.031 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "mon."} : dispatch
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch
2026-02-20T22:40:25.032 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial160:9093"} : dispatch
2026-02-20T22:40:25.033 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:25.033 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.033 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.033 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.033 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch
2026-02-20T22:40:25.033 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://trial160:3000"} : dispatch
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://trial160:9095"} : dispatch
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:25.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:24 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:25.069 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0}
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch apply osd", "all_available_devices": true, "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Marking host: trial160 for OSDSpec preview refresh.
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Marking host: trial192 for OSDSpec preview refresh.
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Saving service osd.all-available-devices spec with placement *
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Reconfiguring ceph-exporter.trial192 (monmap changed)...
2026-02-20T22:40:25.076 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Reconfiguring daemon ceph-exporter.trial192 on trial192
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Reconfiguring crash.trial192 (monmap changed)...
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.crash.trial192", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: Reconfiguring daemon crash.trial192 on trial192
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "mgr.trial192.inkpau", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr services"} : dispatch
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.077 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "mon."} : dispatch
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial160:9093"} : dispatch
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.078 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://trial160:3000"} : dispatch
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://trial160:9095"} : dispatch
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:25.079 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:24 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: Reconfiguring mgr.trial192.inkpau (monmap changed)...
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: Reconfiguring daemon mgr.trial192.inkpau on trial192
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: Reconfiguring mon.trial192 (monmap changed)...
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: Reconfiguring daemon mon.trial192 on trial192
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch
2026-02-20T22:40:26.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial160:9093"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial160:3000"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial160:9095"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: Certificate for "grafana_cert (trial160)" is still valid for 1094 days.
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2950769249' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch
2026-02-20T22:40:26.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:25 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.070 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:26.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: Reconfiguring mgr.trial192.inkpau (monmap changed)...
2026-02-20T22:40:26.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: Reconfiguring daemon mgr.trial192.inkpau on trial192
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: Reconfiguring mon.trial192 (monmap changed)...
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: Reconfiguring daemon mon.trial192 on trial192
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://trial160:9093"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://trial160:3000"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://trial160:9095"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: Certificate for "grafana_cert (trial160)" is still valid for 1094 days.
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:26.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2950769249' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:26.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:26.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch
2026-02-20T22:40:26.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch
2026-02-20T22:40:26.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:25 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:26.188 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:26.410 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:26.448 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":5,"num_osds":0,"num_up_osds":0,"osd_up_since":0,"num_in_osds":0,"osd_in_since":0,"num_remapped_pgs":0}
2026-02-20T22:40:26.815 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:26 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:26.815 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:26 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:26.815 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:26 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3820185535' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:26.846 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:26 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:26.847 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:26 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:40:26.847 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:26 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3820185535' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:27.449 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:27.575 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"} : dispatch
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"}]': finished
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3403062152' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"} : dispatch
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: osdmap e6: 1 total, 0 up, 1 in
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:27.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/666079772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ba86b69-0f7b-4845-98e0-3118956e0f14"} : dispatch
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/666079772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ba86b69-0f7b-4845-98e0-3118956e0f14"}]': finished
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: osdmap e7: 2 total, 0 up, 2 in
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/1379006270' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:28.000 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:27 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3992596445' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:28.069 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:28.082 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"} : dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"}]': finished
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/3403062152' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "db583fef-aeb6-443b-8e83-1771f1f24857"} : dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: osdmap e6: 1 total, 0 up, 1 in
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/666079772' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5ba86b69-0f7b-4845-98e0-3118956e0f14"} : dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/666079772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ba86b69-0f7b-4845-98e0-3118956e0f14"}]': finished
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: osdmap e7: 2 total, 0 up, 2 in
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:28.083 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:28.084 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/1379006270' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:28.084 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:27 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3992596445' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:28.256 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":8,"num_osds":3,"num_up_osds":0,"osd_up_since":0,"num_in_osds":3,"osd_in_since":1771627228,"num_remapped_pgs":0}
2026-02-20T22:40:28.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/4226398804' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"}]': finished
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: osdmap e8: 3 total, 0 up, 3 in
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2382647700' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1317938508' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797"} : dispatch
2026-02-20T22:40:28.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1317938508' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797"}]': finished
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: osdmap e9: 4 total, 0 up, 4 in
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/1868337240' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:28.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:28 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3012050619' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/4226398804' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "05f67c9a-ff34-443c-bae6-11d6aa6da3f0"}]': finished
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: osdmap e8: 3 total, 0 up, 3 in
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2382647700' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1317938508' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1317938508' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797"}]': finished
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: osdmap e9: 4 total, 0 up, 4 in
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/1868337240' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:29.092 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:28 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3012050619' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:29.256 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:29.377 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:29.609 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:29.648 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":11,"num_osds":6,"num_up_osds":0,"osd_up_since":0,"num_in_osds":6,"osd_in_since":1771627229,"num_remapped_pgs":0}
2026-02-20T22:40:30.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3741139182' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"}]': finished
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: osdmap e10: 5 total, 0 up, 5 in
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:30.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/274055198' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "61921929-43bf-4831-beb8-da7c4bfaefd6"} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/274055198' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61921929-43bf-4831-beb8-da7c4bfaefd6"}]': finished
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: osdmap e11: 6 total, 0 up, 6 in
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:30.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:29 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4251320706' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:30.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:30.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"} : dispatch
2026-02-20T22:40:30.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/3741139182' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"} : dispatch
2026-02-20T22:40:30.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "134053b6-7ef3-4bb8-8f52-e40b04e0f044"}]': finished
2026-02-20T22:40:30.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: osdmap e10: 5 total, 0 up, 5 in
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/274055198' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "61921929-43bf-4831-beb8-da7c4bfaefd6"} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/274055198' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61921929-43bf-4831-beb8-da7c4bfaefd6"}]': finished
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: osdmap e11: 6 total, 0 up, 6 in
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:30.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:30.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:30.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:30.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:29 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4251320706' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:30.649 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:30.770 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:30.998 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:30.998 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3901994199' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:30.998 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/775872831' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3160596551' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"}]': finished
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: osdmap e12: 7 total, 0 up, 7 in
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:30.999 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:30 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/3901994199' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/775872831' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/3160596551' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7e16542-8347-441e-ad0f-be5702cbfdb0"}]': finished
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: osdmap e12: 7 total, 0 up, 7 in
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:31.031 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:31.032 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:31.032 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:31.032 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:31.032 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:31.032 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:30 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:31.036 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:32.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:32.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3366551289' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ad3ef80-6c75-4e19-aada-0067bfde8368"} : dispatch
2026-02-20T22:40:32.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3366551289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ad3ef80-6c75-4e19-aada-0067bfde8368"}]': finished
2026-02-20T22:40:32.003 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: osdmap e13: 8 total, 0 up, 8 in
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3692844914' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/1086253569' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:32.004 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:31 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2192777808' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:32.005 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:32.005 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3366551289' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ad3ef80-6c75-4e19-aada-0067bfde8368"} : dispatch
2026-02-20T22:40:32.005 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3366551289' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ad3ef80-6c75-4e19-aada-0067bfde8368"}]': finished
2026-02-20T22:40:32.005 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: osdmap e13: 8 total, 0 up, 8 in
2026-02-20T22:40:32.005 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3692844914' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/1086253569' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:32.006 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:31 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2192777808' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch
2026-02-20T22:40:32.037 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:32.167 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:32.386 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:32.425 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:32.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:32 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch
2026-02-20T22:40:32.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:32 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:32.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:32 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2524827573' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:32.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:32 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch
2026-02-20T22:40:32.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:32 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:32.857 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:32 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch
2026-02-20T22:40:32.857 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:32 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:32.858 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:32 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2524827573' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:32.858 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:32 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch
2026-02-20T22:40:32.858 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:32 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:33.427 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:33.559 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:33.783 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:33.823 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:34.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:33 trial160 ceph-mon[35528]: Deploying daemon osd.0 on trial192
2026-02-20T22:40:34.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:33 trial160 ceph-mon[35528]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:34.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:33 trial160 ceph-mon[35528]: Deploying daemon osd.1 on trial160
2026-02-20T22:40:34.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:33 trial192 ceph-mon[41689]: Deploying daemon osd.0 on trial192
2026-02-20T22:40:34.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:33 trial192 ceph-mon[41689]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:34.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:33 trial192 ceph-mon[41689]: Deploying daemon osd.1 on trial160
2026-02-20T22:40:34.785 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:34 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3726650331' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:34.785 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:34 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:34.785 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:34 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:34.785 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:34 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch
2026-02-20T22:40:34.785 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:34 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:34.825 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:34.948 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:34.981 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:34 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3726650331' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:34.981 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:34 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:34.981 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:34 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:34.981 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:34 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch
2026-02-20T22:40:34.981 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:34 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:35.176 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:35.220 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":13,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: Deploying daemon osd.2 on trial192
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:36.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch
2026-02-20T22:40:36.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch
2026-02-20T22:40:36.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1160844234' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:36.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:35 trial160 ceph-mon[35528]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch
2026-02-20T22:40:36.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: Deploying daemon osd.2 on trial192
2026-02-20T22:40:36.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:36.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1160844234' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:36.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:35 trial192 ceph-mon[41689]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch
2026-02-20T22:40:36.221 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:36.351 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:36.590 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:36.627 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":14,"num_osds":8,"num_up_osds":0,"osd_up_since":0,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: Deploying daemon osd.3 on trial160
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: osdmap e14: 8 total, 0 up, 8 in
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:36.833 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3980653397' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.834 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.835 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch
2026-02-20T22:40:36.835 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:36.835 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:36 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:36.944 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: Deploying daemon osd.3 on trial160
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: osdmap e14: 8 total, 0 up, 8 in
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:36.945 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3980653397' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:36.946 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:36 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:37.629 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: Deploying daemon osd.4 on trial192
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: osdmap e15: 8 total, 0 up, 8 in
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:37.779 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch
2026-02-20T22:40:37.780 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:37 trial160 ceph-mon[35528]: from='osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch
2026-02-20T22:40:37.837 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:37.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: Deploying daemon osd.4 on trial192
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: osdmap e15: 8 total, 0 up, 8 in
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:37.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch
2026-02-20T22:40:37.869 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:37 trial192 ceph-mon[41689]: from='osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch
2026-02-20T22:40:38.081 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:38.120 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":16,"num_osds":8,"num_up_osds":2,"osd_up_since":1771627237,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:38.847 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: Deploying daemon osd.5 on trial160
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741] boot
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817] boot
2026-02-20T22:40:38.848 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: osdmap e16: 8 total, 2 up, 8 in
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:38.849 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:38 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2395764702' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: Deploying daemon osd.5 on trial160
2026-02-20T22:40:39.034 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: osd.1 [v2:10.20.193.160:6802/1999704741,v1:10.20.193.160:6803/1999704741] boot
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: osd.0 [v2:10.20.193.192:6800/1849066817,v1:10.20.193.192:6801/1849066817] boot
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: osdmap e16: 8 total, 2 up, 8 in
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:39.035 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:39.036 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:38 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2395764702' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:39.122 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:39.245 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:39.465 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:39.505 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":17,"num_osds":8,"num_up_osds":2,"osd_up_since":1771627237,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:39.946 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:39.946 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:39.946 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: osdmap e17: 8 total, 2 up, 8 in
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:39.947 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/404951260' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch
2026-02-20T22:40:39.948 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:39 trial160 ceph-mon[35528]: from='osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: osdmap e17: 8 total, 2 up, 8 in
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:40.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:40.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:40.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch
2026-02-20T22:40:40.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:40.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/404951260' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:40.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch
2026-02-20T22:40:40.120 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:39 trial192 ceph-mon[41689]: from='osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch
2026-02-20T22:40:40.506 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:40.627 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:40.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:40.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:40.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: Deploying daemon osd.6 on trial192
2026-02-20T22:40:40.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: Deploying daemon osd.7 on trial160
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162] boot
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924] boot
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: osdmap e18: 8 total, 4 up, 8 in
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:40.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:40.840 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:40.840 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch
2026-02-20T22:40:40.840 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:40 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch
2026-02-20T22:40:40.876 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:40.944 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":19,"num_osds":8,"num_up_osds":4,"osd_up_since":1771627239,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: Deploying daemon osd.6 on trial192
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: Deploying daemon osd.7 on trial160
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: osd.3 [v2:10.20.193.160:6810/3615792162,v1:10.20.193.160:6811/3615792162] boot
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: osd.2 [v2:10.20.193.192:6808/4260818924,v1:10.20.193.192:6809/4260818924] boot
2026-02-20T22:40:41.058 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: osdmap e18: 8 total, 4 up, 8 in
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch
2026-02-20T22:40:41.059 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:40 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch
2026-02-20T22:40:41.946 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:42.008 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: pgmap v23: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: osdmap e19: 8 total, 4 up, 8 in
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:42.009 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1830054047' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.010 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:41 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.083 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: pgmap v23: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: osdmap e19: 8 total, 4 up, 8 in
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:42.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1830054047' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:41 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:42.324 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:42.370 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":20,"num_osds":8,"num_up_osds":5,"osd_up_since":1771627241,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished
2026-02-20T22:40:43.082 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503] boot
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: osdmap e20: 8 total, 5 up, 8 in
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3432910350' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.083 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:42 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: osd.4 [v2:10.20.193.192:6816/3376545503,v1:10.20.193.192:6817/3376545503] boot
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: osdmap e20: 8 total, 5 up, 8 in
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3432910350' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.087 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:42 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.371 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:43.493 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:43.722 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:43.761 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":21,"num_osds":8,"num_up_osds":6,"osd_up_since":1771627242,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":1}
2026-02-20T22:40:43.975 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: pgmap v26: 1 pgs: 1 unknown; 0 B data, 132 MiB used, 3.4 TiB / 3.4 TiB avail
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734] boot
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: osdmap e21: 8 total, 6 up, 8 in
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:43.976 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:43.977 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:43 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3077935712' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:44.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: pgmap v26: 1 pgs: 1 unknown; 0 B data, 132 MiB used, 3.4 TiB / 3.4 TiB avail
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial192", "root=default"]}]': finished
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: osd.5 [v2:10.20.193.160:6818/1363388734,v1:10.20.193.160:6819/1363388734] boot
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: osdmap e21: 8 total, 6 up, 8 in
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial160", "root=default"]} : dispatch
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:40:44.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:44.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:43 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3077935712' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:44.762 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd stat -f json
2026-02-20T22:40:44.886 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082] boot
2026-02-20T22:40:45.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: osdmap e22: 8 total, 7 up, 8 in
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:45.090 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:44 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:45.102 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:45.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:45.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:45.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial160", "root=default"]}]': finished
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: osd.6 [v2:10.20.193.192:6824/83502082,v1:10.20.193.192:6825/83502082] boot
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: osdmap e22: 8 total, 7 up, 8 in
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:45.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:44 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:45.140 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1771627244,"num_in_osds":8,"osd_in_since":1771627230,"num_remapped_pgs":0}
2026-02-20T22:40:45.141 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd dump --format=json
2026-02-20T22:40:45.257 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:45.474 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:45.474 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":23,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","created":"2026-02-20T22:39:24.775520+0000","modified":"2026-02-20T22:40:44.839656+0000","last_up_change":"2026-02-20T22:40:44.839656+0000","last_in_change":"2026-02-20T22:40:30.937065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":11,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T22:40:40.745119+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db583fef-aeb6-443b-8e83-1771f1f24857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6800","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6801","nonce":1849066817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6802","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6803","nonce":1849066817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6806","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6807","nonce":1849066817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6804","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6805","nonce":1849066817}]},"public_addr":"10.20.193.192:6801/1849066817","cluster_addr":"10.20.193.192:6803/1849066817","heartbeat_back_addr":"10.20.193.192:6807/1849066817","heartbeat_front_addr":"10.20.193.192:6805/1849066817","state":["exists","up"]},{"osd":1,"uuid":"5ba86b69-0f7b-4845-98e0-3118956e0f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6803","nonce":1999704741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6804","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6805","nonce":1999704741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6809","nonce":1999704741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6807","nonce":1999704741}]},"public_addr":"10.20.193.160:6803/1999704741","cluster_addr":"10.20.193.160:6805/1999704741","heartbeat_back_addr":"10.20.193.160:6809/1999704741","heartbeat_front_addr":"10.20.193.160:6807/1999704741","state":["exists","up"]},{"osd":2,"uuid":"05f67c9a-ff34-443c-bae6-11d6aa6da3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6808","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6809","nonce":4260818924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6810","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6811","nonce":4260818924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6814","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6815","nonce":4260818924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6812","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6813","nonce":4260818924}]},"public_addr":"10.20.193.192:6809/4260818924","cluster_addr":"10.20.193.192:6811/4260818924","heartbeat_back_addr":"10.20.193.192:6815/4260818924","heartbeat_front_addr":"10.20.193.192:6813/4260818924","state":["exists","up"]},{"osd":3,"uuid":"5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6810","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6811","nonce":3615792162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6812","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6813","nonce":3615792162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6816","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6817","nonce":3615792162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6814","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6815","nonce":3615792162}]},"public_addr":"10.20.193.160:6811/3615792162","cluster_addr":"10.20.193.160:6813/3615792162","heartbeat_back_addr":"10.20.193.160:6817/3615792162","heartbeat_front_addr":"10.20.193.160:6815/3615792162","state":["exists","up"]},{"osd":4,"uuid":"134053b6-7ef3-4bb8-8f52-e40b04e0f044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6816","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6817","nonce":3376545503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6818","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6819","nonce":3376545503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6822","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6823","nonce":3376545503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6820","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6821","nonce":3376545503}]},"public_addr":"10.20.193.192:6817/3376545503","cluster_addr":"10.20.193.192:6819/3376545503","heartbeat_back_addr":"10.20.193.192:6823/3376545503","heartbeat_front_addr":"10.20.193.192:6821/3376545503","state":["exists","up"]},{"osd":5,"uuid":"61921929-43bf-4831-beb8-da7c4bfaefd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6818","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6819","nonce":1363388734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6820","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6821","nonce":1363388734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6824","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6825","nonce":1363388734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6822","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6823","nonce":1363388734}]},"public_addr":"10.20.193.160:6819/1363388734","cluster_addr":"10.20.193.160:6821/1363388734","heartbeat_back_addr":"10.20.193.160:6825/1363388734","heartbeat_front_addr":"10.20.193.160:6823/1363388734","state":["exists","up"]},{"osd":6,"uuid":"e7e16542-8347-441e-ad0f-be5702cbfdb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6824","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6825","nonce":83502082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6826","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6827","nonce":83502082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6830","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6831","nonce":83502082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6828","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6829","nonce":83502082}]},"public_addr":"10.20.193.192:6825/83502082","cluster_addr":"10.20.193.192:6827/83502082","heartbeat_back_addr":"10.20.193.192:6831/83502082","heartbeat_front_addr":"10.20.193.192:6829/83502082","state":["exists","up"]},{"osd":7,"uuid":"3ad3ef80-6c75-4e19-aada-0067bfde8368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6826","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6827","nonce":251729473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6828","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6829","nonce":251729473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6832","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6833","nonce":251729473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6830","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6831","nonce":251729473}]},"public_addr":"10.20.193.160:6827/251729473","cluster_addr":"10.20.193.160:6829/251729473","heartbeat_back_addr":"10.20.193.160:6833/251729473","heartbeat_front_addr":"10.20.193.160:6831/251729473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.125037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.530281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.469580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.762645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:40.617100+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:41.010840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:42.833323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.160:6800/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/3187096191":"2026-02-21T22:40:06.540069+0000","10.20.193.160:6801/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/881847967":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2228701640":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/3157666537":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/2455371783":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/278762300":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2637985551":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/629551054":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/1599508014":"2026-02-21T22:40:06.540069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}}
2026-02-20T22:40:45.512 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-20T22:40:40.745119+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}]
2026-02-20T22:40:45.512 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd pool get .mgr pg_num
2026-02-20T22:40:45.627 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:45.845 INFO:teuthology.orchestra.run.trial160.stdout:pg_num: 1
2026-02-20T22:40:45.884 INFO:tasks.cephadm:Setting up client nodes...
2026-02-20T22:40:45.885 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *'
2026-02-20T22:40:45.908 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: purged_snaps scrub starts
2026-02-20T22:40:45.908 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: purged_snaps scrub ok
2026-02-20T22:40:45.908 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: Detected new or changed devices on trial192
2026-02-20T22:40:45.908 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: Adjusting osd_memory_target on trial192 to 20467M
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: pgmap v29: 1 pgs: 1 unknown; 0 B data, 186 MiB used, 4.8 TiB / 4.8 TiB avail
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: Detected new or changed devices on trial160
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: Adjusting osd_memory_target on trial160 to 19699M
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473] boot
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: osdmap e23: 8 total, 8 up, 8 in
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3268885457' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:45.909 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:45 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4092225387' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:46.002 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:46.107 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: purged_snaps scrub starts
2026-02-20T22:40:46.107 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: purged_snaps scrub ok
2026-02-20T22:40:46.107 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: Detected new or changed devices on trial192
2026-02-20T22:40:46.107 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: Adjusting osd_memory_target on trial192 to 20467M
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: pgmap v29: 1 pgs: 1 unknown; 0 B data, 186 MiB used, 4.8 TiB / 4.8 TiB avail
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: Detected new or changed devices on trial160
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: Adjusting osd_memory_target on trial160 to 19699M
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: osd.7 [v2:10.20.193.160:6826/251729473,v1:10.20.193.160:6827/251729473] boot
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: osdmap e23: 8 total, 8 up, 8 in
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3268885457' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch
2026-02-20T22:40:46.108 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:45 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4092225387' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:46.282 INFO:teuthology.orchestra.run.trial160.stdout:[client.0]
2026-02-20T22:40:46.282 INFO:teuthology.orchestra.run.trial160.stdout:	key = AQDu4phpoFGWEBAAVyLkV/CSnlfJsXf29rjUKg==
2026-02-20T22:40:46.320 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:40:46.321 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/etc/ceph/ceph.client.0.keyring
2026-02-20T22:40:46.321 DEBUG:teuthology.orchestra.run.trial160:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring
2026-02-20T22:40:46.398 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *'
2026-02-20T22:40:46.522 INFO:teuthology.orchestra.run.trial192.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial192/config
2026-02-20T22:40:46.579 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68945]:     ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1
2026-02-20T22:40:46.579 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68945]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68945]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167)
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68945]: pam_unix(sudo:session): session closed for user root
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68960]:     ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68960]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68960]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167)
2026-02-20T22:40:46.580 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 sudo[68960]: pam_unix(sudo:session): session closed for user root
2026-02-20T22:40:46.800 INFO:teuthology.orchestra.run.trial192.stdout:[client.1]
2026-02-20T22:40:46.800 INFO:teuthology.orchestra.run.trial192.stdout:	key = AQDu4phpc9dVLxAADkbRGHsLHd5l4N6tBNYX5A==
2026-02-20T22:40:46.800 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57129]:     ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1
2026-02-20T22:40:46.800 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57129]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory
2026-02-20T22:40:46.800 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57129]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167)
2026-02-20T22:40:46.801 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57129]: pam_unix(sudo:session): session closed for user root
2026-02-20T22:40:46.801 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57135]:     ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1
2026-02-20T22:40:46.801 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory
2026-02-20T22:40:46.801 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57135]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167)
2026-02-20T22:40:46.801 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 sudo[57135]: pam_unix(sudo:session): session closed for user root
2026-02-20T22:40:46.837 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:40:46.837 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/etc/ceph/ceph.client.1.keyring
2026-02-20T22:40:46.837 DEBUG:teuthology.orchestra.run.trial192:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring
2026-02-20T22:40:46.874 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph config log 1 --format=json
2026-02-20T22:40:46.989 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: osdmap e24: 8 total, 8 up, 8 in
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/134455887' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4220391676' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.105 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4220391676' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished
2026-02-20T22:40:47.106 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.107 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: pgmap v32: 1 pgs: 1 unknown; 0 B data, 212 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:47.107 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.107 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished
2026-02-20T22:40:47.107 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:46 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/3963435407' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: osdmap e24: 8 total, 8 up, 8 in
2026-02-20T22:40:47.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/134455887' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch
2026-02-20T22:40:47.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4220391676' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4220391676' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:47.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: pgmap v32: 1 pgs: 1 unknown; 0 B data, 212 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished
2026-02-20T22:40:47.119 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:46 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/3963435407' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch
2026-02-20T22:40:47.205 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:47.241 INFO:teuthology.orchestra.run.trial160.stdout:[{"version":15,"timestamp":"2026-02-20T22:40:44.817702+0000","name":"","changes":[{"name":"osd/host:trial160/osd_memory_target","new_value":"20656749772"}]}]
2026-02-20T22:40:47.242 INFO:tasks.ceph_manager:config epoch is 15
2026-02-20T22:40:47.242 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean...
2026-02-20T22:40:47.242 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available
2026-02-20T22:40:47.242 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mgr dump --format=json
2026-02-20T22:40:47.361 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:47.612 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:47.650 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":19,"flags":0,"active_gid":14213,"active_name":"trial160.pdmeos","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6800","nonce":2245479657},{"type":"v1","addr":"10.20.193.160:6801","nonce":2245479657}]},"active_addr":"10.20.193.160:6801/2245479657","active_change":"2026-02-20T22:40:06.540168+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[{"gid":14236,"name":"trial192.inkpau","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace.  Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self certificates generated and signed by cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.5","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"how frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB.  Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace.  Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self certificates generated and signed by cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.5","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:ceph20-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"how frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB.  Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://10.20.193.160:8443/","prometheus":"http://10.20.193.160:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.160:0","nonce":1767220260}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.160:0","nonce":3378995981}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.160:0","nonce":806948305}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.160:0","nonce":379751881}]}]}
2026-02-20T22:40:47.658 INFO:tasks.cephadm.ceph_manager.ceph:mgr available!
2026-02-20T22:40:47.658 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up
2026-02-20T22:40:47.659 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd dump --format=json
2026-02-20T22:40:47.776 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:47.891 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:47 trial160 ceph-mon[35528]: osdmap e25: 8 total, 8 up, 8 in
2026-02-20T22:40:47.891 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:47 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3284723775' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch
2026-02-20T22:40:47.891 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:47 trial160 ceph-mon[35528]: mgrmap e19: trial160.pdmeos(active, since 40s), standbys: trial192.inkpau
2026-02-20T22:40:47.891 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:47 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2427778243' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch
2026-02-20T22:40:48.000 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:48.000 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":25,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","created":"2026-02-20T22:39:24.775520+0000","modified":"2026-02-20T22:40:46.841925+0000","last_up_change":"2026-02-20T22:40:44.839656+0000","last_in_change":"2026-02-20T22:40:30.937065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":11,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T22:40:40.745119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db583fef-aeb6-443b-8e83-1771f1f24857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6800","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6801","nonce":1849066817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6802","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6803","nonce":1849066817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6806","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6807","nonce":1849066817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6804","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6805","nonce":1849066817}]},"public_addr":"10.20.193.192:6801/1849066817","cluster_addr":"10.20.193.192:6803/1849066817","heartbeat_back_addr":"10.20.193.192:6807/1849066817","heartbeat_front_addr":"10.20.193.192:6805/1849066817","state":["exists","up"]},{"osd":1,"uuid":"5ba86b69-0f7b-4845-98e0-3118956e0f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6803","nonce":1999704741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6804","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6805","nonce":1999704741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6809","nonce":1999704741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6807","nonce":1999704741}]},"public_addr":"10.20.193.160:6803/1999704741","cluster_addr":"10.20.193.160:6805/1999704741","heartbeat_back_addr":"10.20.193.160:6809/1999704741","heartbeat_front_addr":"10.20.193.160:6807/1999704741","state":["exists","up"]},{"osd":2,"uuid":"05f67c9a-ff34-443c-bae6-11d6aa6da3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6808","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6809","nonce":4260818924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6810","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6811","nonce":4260818924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6814","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6815","nonce":4260818924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6812","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6813","nonce":4260818924}]},"public_addr":"10.20.193.192:6809/4260818924","cluster_addr":"10.20.193.192:6811/4260818924","heartbeat_back_addr":"10.20.193.192:6815/4260818924","heartbeat_front_addr":"10.20.193.192:6813/4260818924","state":["exists","up"]},{"osd":3,"uuid":"5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6810","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6811","nonce":3615792162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6812","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6813","nonce":3615792162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6816","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6817","nonce":3615792162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6814","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6815","nonce":3615792162}]},"public_addr":"10.20.193.160:6811/3615792162","cluster_addr":"10.20.193.160:6813/3615792162","heartbeat_back_addr":"10.20.193.160:6817/3615792162","heartbeat_front_addr":"10.20.193.160:6815/3615792162","state":["exists","up"]},{"osd":4,"uuid":"134053b6-7ef3-4bb8-8f52-e40b04e0f044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6816","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6817","nonce":3376545503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6818","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6819","nonce":3376545503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6822","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6823","nonce":3376545503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6820","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6821","nonce":3376545503}]},"public_addr":"10.20.193.192:6817/3376545503","cluster_addr":"10.20.193.192:6819/3376545503","heartbeat_back_addr":"10.20.193.192:6823/3376545503","heartbeat_front_addr":"10.20.193.192:6821/3376545503","state":["exists","up"]},{"osd":5,"uuid":"61921929-43bf-4831-beb8-da7c4bfaefd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6818","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6819","nonce":1363388734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6820","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6821","nonce":1363388734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6824","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6825","nonce":1363388734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6822","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6823","nonce":1363388734}]},"public_addr":"10.20.193.160:6819/1363388734","cluster_addr":"10.20.193.160:6821/1363388734","heartbeat_back_addr":"10.20.193.160:6825/1363388734","heartbeat_front_addr":"10.20.193.160:6823/1363388734","state":["exists","up"]},{"osd":6,"uuid":"e7e16542-8347-441e-ad0f-be5702cbfdb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6824","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6825","nonce":83502082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6826","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6827","nonce":83502082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6830","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6831","nonce":83502082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6828","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6829","nonce":83502082}]},"public_addr":"10.20.193.192:6825/83502082","cluster_addr":"10.20.193.192:6827/83502082","heartbeat_back_addr":"10.20.193.192:6831/83502082","heartbeat_front_addr":"10.20.193.192:6829/83502082","state":["exists","up"]},{"osd":7,"uuid":"3ad3ef80-6c75-4e19-aada-0067bfde8368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6826","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6827","nonce":251729473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6828","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6829","nonce":251729473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6832","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6833","nonce":251729473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6830","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6831","nonce":251729473}]},"public_addr":"10.20.193.160:6827/251729473","cluster_addr":"10.20.193.160:6829/251729473","heartbeat_back_addr":"10.20.193.160:6833/251729473","heartbeat_front_addr":"10.20.193.160:6831/251729473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.125037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.530281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.469580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.762645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:40.617100+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:41.010840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:42.833323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:43.187724+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.160:6800/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/3187096191":"2026-02-21T22:40:06.540069+0000","10.20.193.160:6801/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/881847967":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2228701640":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/3157666537":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/2455371783":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/278762300":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2637985551":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/629551054":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/1599508014":"2026-02-21T22:40:06.540069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}}
2026-02-20T22:40:48.038 INFO:tasks.cephadm.ceph_manager.ceph:all up!
2026-02-20T22:40:48.039 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd dump --format=json
2026-02-20T22:40:48.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:47 trial192 ceph-mon[41689]: osdmap e25: 8 total, 8 up, 8 in
2026-02-20T22:40:48.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:47 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3284723775' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch
2026-02-20T22:40:48.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:47 trial192 ceph-mon[41689]: mgrmap e19: trial160.pdmeos(active, since 40s), standbys: trial192.inkpau
2026-02-20T22:40:48.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:47 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2427778243' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch
2026-02-20T22:40:48.154 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.369 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:48.369 INFO:teuthology.orchestra.run.trial160.stdout:{"epoch":25,"fsid":"fa4d44c0-0eac-11f1-86c6-d404e6e7d460","created":"2026-02-20T22:39:24.775520+0000","modified":"2026-02-20T22:40:46.841925+0000","last_up_change":"2026-02-20T22:40:44.839656+0000","last_in_change":"2026-02-20T22:40:30.937065+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":11,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T22:40:40.745119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"db583fef-aeb6-443b-8e83-1771f1f24857","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6800","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6801","nonce":1849066817}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6802","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6803","nonce":1849066817}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6806","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6807","nonce":1849066817}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6804","nonce":1849066817},{"type":"v1","addr":"10.20.193.192:6805","nonce":1849066817}]},"public_addr":"10.20.193.192:6801/1849066817","cluster_addr":"10.20.193.192:6803/1849066817","heartbeat_back_addr":"10.20.193.192:6807/1849066817","heartbeat_front_addr":"10.20.193.192:6805/1849066817","state":["exists","up"]},{"osd":1,"uuid":"5ba86b69-0f7b-4845-98e0-3118956e0f14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6802","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6803","nonce":1999704741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6804","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6805","nonce":1999704741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6808","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6809","nonce":1999704741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6806","nonce":1999704741},{"type":"v1","addr":"10.20.193.160:6807","nonce":1999704741}]},"public_addr":"10.20.193.160:6803/1999704741","cluster_addr":"10.20.193.160:6805/1999704741","heartbeat_back_addr":"10.20.193.160:6809/1999704741","heartbeat_front_addr":"10.20.193.160:6807/1999704741","state":["exists","up"]},{"osd":2,"uuid":"05f67c9a-ff34-443c-bae6-11d6aa6da3f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6808","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6809","nonce":4260818924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6810","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6811","nonce":4260818924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6814","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6815","nonce":4260818924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6812","nonce":4260818924},{"type":"v1","addr":"10.20.193.192:6813","nonce":4260818924}]},"public_addr":"10.20.193.192:6809/4260818924","cluster_addr":"10.20.193.192:6811/4260818924","heartbeat_back_addr":"10.20.193.192:6815/4260818924","heartbeat_front_addr":"10.20.193.192:6813/4260818924","state":["exists","up"]},{"osd":3,"uuid":"5818e2fd-8a65-4e2b-b3f7-aafc0f8a8797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6810","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6811","nonce":3615792162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6812","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6813","nonce":3615792162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6816","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6817","nonce":3615792162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6814","nonce":3615792162},{"type":"v1","addr":"10.20.193.160:6815","nonce":3615792162}]},"public_addr":"10.20.193.160:6811/3615792162","cluster_addr":"10.20.193.160:6813/3615792162","heartbeat_back_addr":"10.20.193.160:6817/3615792162","heartbeat_front_addr":"10.20.193.160:6815/3615792162","state":["exists","up"]},{"osd":4,"uuid":"134053b6-7ef3-4bb8-8f52-e40b04e0f044","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":20,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6816","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6817","nonce":3376545503}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6818","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6819","nonce":3376545503}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6822","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6823","nonce":3376545503}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6820","nonce":3376545503},{"type":"v1","addr":"10.20.193.192:6821","nonce":3376545503}]},"public_addr":"10.20.193.192:6817/3376545503","cluster_addr":"10.20.193.192:6819/3376545503","heartbeat_back_addr":"10.20.193.192:6823/3376545503","heartbeat_front_addr":"10.20.193.192:6821/3376545503","state":["exists","up"]},{"osd":5,"uuid":"61921929-43bf-4831-beb8-da7c4bfaefd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6818","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6819","nonce":1363388734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6820","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6821","nonce":1363388734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6824","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6825","nonce":1363388734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6822","nonce":1363388734},{"type":"v1","addr":"10.20.193.160:6823","nonce":1363388734}]},"public_addr":"10.20.193.160:6819/1363388734","cluster_addr":"10.20.193.160:6821/1363388734","heartbeat_back_addr":"10.20.193.160:6825/1363388734","heartbeat_front_addr":"10.20.193.160:6823/1363388734","state":["exists","up"]},{"osd":6,"uuid":"e7e16542-8347-441e-ad0f-be5702cbfdb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6824","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6825","nonce":83502082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6826","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6827","nonce":83502082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6830","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6831","nonce":83502082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.192:6828","nonce":83502082},{"type":"v1","addr":"10.20.193.192:6829","nonce":83502082}]},"public_addr":"10.20.193.192:6825/83502082","cluster_addr":"10.20.193.192:6827/83502082","heartbeat_back_addr":"10.20.193.192:6831/83502082","heartbeat_front_addr":"10.20.193.192:6829/83502082","state":["exists","up"]},{"osd":7,"uuid":"3ad3ef80-6c75-4e19-aada-0067bfde8368","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6826","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6827","nonce":251729473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6828","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6829","nonce":251729473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6832","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6833","nonce":251729473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.160:6830","nonce":251729473},{"type":"v1","addr":"10.20.193.160:6831","nonce":251729473}]},"public_addr":"10.20.193.160:6827/251729473","cluster_addr":"10.20.193.160:6829/251729473","heartbeat_back_addr":"10.20.193.160:6833/251729473","heartbeat_front_addr":"10.20.193.160:6831/251729473","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.125037+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:36.530281+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.469580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:38.762645+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:40.617100+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:41.010840+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:42.833323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-20T22:40:43.187724+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.160:6800/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/3187096191":"2026-02-21T22:40:06.540069+0000","10.20.193.160:6801/2178254940":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/881847967":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2228701640":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/3157666537":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/2455371783":"2026-02-21T22:40:06.540069+0000","10.20.193.160:0/278762300":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:6800/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/3517766381":"2026-02-21T22:39:34.696418+0000","10.20.193.160:0/2637985551":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/629551054":"2026-02-21T22:39:45.779748+0000","10.20.193.160:6801/84586591":"2026-02-21T22:39:45.779748+0000","10.20.193.160:0/1599508014":"2026-02-21T22:40:06.540069+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}}
2026-02-20T22:40:48.407 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats
2026-02-20T22:40:48.407 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats
2026-02-20T22:40:48.408 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats
2026-02-20T22:40:48.408 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats
2026-02-20T22:40:48.409 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.4 flush_pg_stats
2026-02-20T22:40:48.409 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.5 flush_pg_stats
2026-02-20T22:40:48.409 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.6 flush_pg_stats
2026-02-20T22:40:48.410 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph tell osd.7 flush_pg_stats
2026-02-20T22:40:48.537 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.537 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.538 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.539 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.545 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.546 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.547 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.571 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.742 INFO:teuthology.orchestra.run.trial160.stdout:98784247811
2026-02-20T22:40:48.742 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.7
2026-02-20T22:40:48.744 INFO:teuthology.orchestra.run.trial160.stdout:68719476739
2026-02-20T22:40:48.744 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.1
2026-02-20T22:40:48.752 INFO:teuthology.orchestra.run.trial160.stdout:68719476739
2026-02-20T22:40:48.752 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.0
2026-02-20T22:40:48.755 INFO:teuthology.orchestra.run.trial160.stdout:90194313220
2026-02-20T22:40:48.755 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.5
2026-02-20T22:40:48.761 INFO:teuthology.orchestra.run.trial160.stdout:94489280515
2026-02-20T22:40:48.761 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.6
2026-02-20T22:40:48.763 INFO:teuthology.orchestra.run.trial160.stdout:85899345923
2026-02-20T22:40:48.764 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.4
2026-02-20T22:40:48.772 INFO:teuthology.orchestra.run.trial160.stdout:77309411332
2026-02-20T22:40:48.772 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.3
2026-02-20T22:40:48.774 INFO:teuthology.orchestra.run.trial160.stdout:77309411332
2026-02-20T22:40:48.775 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.2
2026-02-20T22:40:48.866 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.878 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.878 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.883 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.892 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.893 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.938 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:48 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/95033200' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:48.938 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:48 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3776235059' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:48.938 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:48 trial160 ceph-mon[35528]: pgmap v34: 1 pgs: 1 unknown; 0 B data, 213 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:48.941 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:48.942 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:49.105 INFO:teuthology.orchestra.run.trial160.stdout:98784247810
2026-02-20T22:40:49.109 INFO:teuthology.orchestra.run.trial160.stdout:68719476738
2026-02-20T22:40:49.110 INFO:teuthology.orchestra.run.trial160.stdout:68719476738
2026-02-20T22:40:49.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:48 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/95033200' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:49.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:48 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3776235059' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch
2026-02-20T22:40:49.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:48 trial192 ceph-mon[41689]: pgmap v34: 1 pgs: 1 unknown; 0 B data, 213 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:49.123 INFO:teuthology.orchestra.run.trial160.stdout:85899345922
2026-02-20T22:40:49.125 INFO:teuthology.orchestra.run.trial160.stdout:90194313219
2026-02-20T22:40:49.127 INFO:teuthology.orchestra.run.trial160.stdout:94489280514
2026-02-20T22:40:49.154 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.7
2026-02-20T22:40:49.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476738 for osd.1
2026-02-20T22:40:49.174 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345923 got 85899345922 for osd.4
2026-02-20T22:40:49.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.6
2026-02-20T22:40:49.183 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476738 for osd.0
2026-02-20T22:40:49.190 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.5
2026-02-20T22:40:49.191 INFO:teuthology.orchestra.run.trial160.stdout:77309411331
2026-02-20T22:40:49.192 INFO:teuthology.orchestra.run.trial160.stdout:77309411331
2026-02-20T22:40:49.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.3
2026-02-20T22:40:49.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.2
2026-02-20T22:40:50.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/467293938' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:50.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1712400085' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:50.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3341908728' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:50.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1535099621' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:50.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1838087753' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:50.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1771085660' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:50.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/372567703' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:50.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:49 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1694855335' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:50.155 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.7
2026-02-20T22:40:50.158 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.1
2026-02-20T22:40:50.175 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.4
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/467293938' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1712400085' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3341908728' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1535099621' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1838087753' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:50.178 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1771085660' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:50.179 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/372567703' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:50.179 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:49 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1694855335' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:50.180 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.6
2026-02-20T22:40:50.184 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.0
2026-02-20T22:40:50.191 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.5
2026-02-20T22:40:50.236 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.3
2026-02-20T22:40:50.237 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.2
2026-02-20T22:40:50.274 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.296 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.297 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.298 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.346 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.347 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.393 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.393 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:50.522 INFO:teuthology.orchestra.run.trial160.stdout:98784247810
2026-02-20T22:40:50.522 INFO:teuthology.orchestra.run.trial160.stdout:85899345922
2026-02-20T22:40:50.523 INFO:teuthology.orchestra.run.trial160.stdout:68719476738
2026-02-20T22:40:50.529 INFO:teuthology.orchestra.run.trial160.stdout:94489280514
2026-02-20T22:40:50.575 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345923 got 85899345922 for osd.4
2026-02-20T22:40:50.580 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476738 for osd.1
2026-02-20T22:40:50.582 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280515 got 94489280514 for osd.6
2026-02-20T22:40:50.586 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247811 got 98784247810 for osd.7
2026-02-20T22:40:50.590 INFO:teuthology.orchestra.run.trial160.stdout:68719476738
2026-02-20T22:40:50.591 INFO:teuthology.orchestra.run.trial160.stdout:90194313219
2026-02-20T22:40:50.638 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.5
2026-02-20T22:40:50.638 INFO:teuthology.orchestra.run.trial160.stdout:77309411331
2026-02-20T22:40:50.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476738 for osd.0
2026-02-20T22:40:50.642 INFO:teuthology.orchestra.run.trial160.stdout:77309411331
2026-02-20T22:40:50.677 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.3
2026-02-20T22:40:50.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411331 for osd.2
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/692788220' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1684558364' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4292582020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1031214026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/4126746548' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3942701339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/251880060' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:51.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:50 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2834560626' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/692788220' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1684558364' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4292582020' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1031214026' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/4126746548' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3942701339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/251880060' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:51.338 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:50 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2834560626' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:51.576 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.4
2026-02-20T22:40:51.581 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.1
2026-02-20T22:40:51.583 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.6
2026-02-20T22:40:51.586 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.7
2026-02-20T22:40:51.638 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.5
2026-02-20T22:40:51.641 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.0
2026-02-20T22:40:51.678 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.3
2026-02-20T22:40:51.684 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph osd last-stat-seq osd.2
2026-02-20T22:40:51.697 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.739 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.741 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.743 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.787 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.789 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.825 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.825 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:51.945 INFO:teuthology.orchestra.run.trial160.stdout:85899345923
2026-02-20T22:40:51.975 INFO:teuthology.orchestra.run.trial160.stdout:94489280516
2026-02-20T22:40:51.975 INFO:teuthology.orchestra.run.trial160.stdout:98784247812
2026-02-20T22:40:51.983 INFO:teuthology.orchestra.run.trial160.stdout:68719476740
2026-02-20T22:40:51.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 85899345923 got 85899345923 for osd.4
2026-02-20T22:40:51.987 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 94489280515 got 94489280516 for osd.6
2026-02-20T22:40:52.024 INFO:teuthology.orchestra.run.trial160.stdout:68719476740
2026-02-20T22:40:52.024 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.026 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247811 got 98784247812 for osd.7
2026-02-20T22:40:52.026 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.026 INFO:teuthology.orchestra.run.trial160.stdout:90194313220
2026-02-20T22:40:52.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476740 for osd.1
2026-02-20T22:40:52.030 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.064 INFO:teuthology.orchestra.run.trial160.stdout:77309411332
2026-02-20T22:40:52.068 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476739 got 68719476740 for osd.0
2026-02-20T22:40:52.068 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.068 INFO:teuthology.orchestra.run.trial160.stdout:77309411332
2026-02-20T22:40:52.069 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.5
2026-02-20T22:40:52.069 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.102 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:51 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:52.103 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411332 for osd.2
2026-02-20T22:40:52.103 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411332 got 77309411332 for osd.3
2026-02-20T22:40:52.109 DEBUG:teuthology.parallel:result is None
2026-02-20T22:40:52.109 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean
2026-02-20T22:40:52.109 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph pg dump --format=json
2026-02-20T22:40:52.116 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:51 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:40:52.262 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:52.473 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:52.473 INFO:teuthology.orchestra.run.trial160.stderr:dumped all
2026-02-20T22:40:52.510 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2026-02-20T22:40:50.547699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":5871992832,"kb_used":219904,"kb_used_data":3628,"kb_used_omap":61,"kb_used_meta":215490,"kb_avail":5871772928,"statfs":{"total":6012920659968,"available":6012695478272,"internally_reserved":0,"allocated":3715072,"data_stored":2497536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63028,"internal_metadata":220662220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"24'232","reported_seq":288,"reported_epoch":25,"state":"active+clean","last_fresh":"2026-02-20T22:40:47.387229+0000","last_change":"2026-02-20T22:40:45.979115+0000","last_active":"2026-02-20T22:40:47.387229+0000","last_peered":"2026-02-20T22:40:47.387229+0000","last_clean":"2026-02-20T22:40:47.387229+0000","last_became_active":"2026-02-20T22:40:45.978932+0000","last_became_peered":"2026-02-20T22:40:45.978932+0000","last_unstale":"2026-02-20T22:40:47.387229+0000","last_undegraded":"2026-02-20T22:40:47.387229+0000","last_fullsized":"2026-02-20T22:40:47.387229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T22:40:40.835769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T22:40:40.835769+0000","last_clean_scrub_stamp":"2026-02-20T22:40:40.835769+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T04:15:35.342958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":5,"kb_used_meta":26938,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5514,"internal_metadata":27585142},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6167,"internal_metadata":27584489},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":20,"seq":85899345923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411332,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":18,"seq":77309411332,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8773,"internal_metadata":27581883},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10074,"internal_metadata":27580582},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}}
2026-02-20T22:40:52.510 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph pg dump --format=json
2026-02-20T22:40:52.624 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:52.834 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:52.835 INFO:teuthology.orchestra.run.trial160.stderr:dumped all
2026-02-20T22:40:52.871 INFO:teuthology.orchestra.run.trial160.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2026-02-20T22:40:52.548009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":5871992832,"kb_used":219904,"kb_used_data":3628,"kb_used_omap":61,"kb_used_meta":215490,"kb_avail":5871772928,"statfs":{"total":6012920659968,"available":6012695478272,"internally_reserved":0,"allocated":3715072,"data_stored":2497536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":63028,"internal_metadata":220662220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000310"},"pg_stats":[{"pgid":"1.0","version":"24'232","reported_seq":288,"reported_epoch":25,"state":"active+clean","last_fresh":"2026-02-20T22:40:47.387229+0000","last_change":"2026-02-20T22:40:45.979115+0000","last_active":"2026-02-20T22:40:47.387229+0000","last_peered":"2026-02-20T22:40:47.387229+0000","last_clean":"2026-02-20T22:40:47.387229+0000","last_became_active":"2026-02-20T22:40:45.978932+0000","last_became_peered":"2026-02-20T22:40:45.978932+0000","last_unstale":"2026-02-20T22:40:47.387229+0000","last_undegraded":"2026-02-20T22:40:47.387229+0000","last_fullsized":"2026-02-20T22:40:47.387229+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T22:40:40.835769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T22:40:40.835769+0000","last_clean_scrub_stamp":"2026-02-20T22:40:40.835769+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-22T04:15:35.342958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":146,"num_read_kb":127,"num_write":277,"num_write_kb":5804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":23,"seq":98784247812,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":5,"kb_used_meta":26938,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5514,"internal_metadata":27585142},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280516,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6167,"internal_metadata":27584489},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6825,"internal_metadata":27583831},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":20,"seq":85899345924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411333,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":733999104,"kb_used":27968,"kb_used_data":816,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733971136,"statfs":{"total":751615082496,"available":751586443264,"internally_reserved":0,"allocated":835584,"data_stored":681172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8775,"internal_metadata":27581881},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":18,"seq":77309411333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":8,"kb_used_meta":26935,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8773,"internal_metadata":27581883},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10074,"internal_metadata":27580582},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":733999104,"kb_used":27200,"kb_used_data":236,"kb_used_omap":9,"kb_used_meta":26934,"kb_avail":733971904,"statfs":{"total":751615082496,"available":751587229696,"internally_reserved":0,"allocated":241664,"data_stored":90804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9425,"internal_metadata":27581231},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}}
2026-02-20T22:40:52.872 INFO:tasks.cephadm.ceph_manager.ceph:clean!
2026-02-20T22:40:52.872 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy...
2026-02-20T22:40:52.872 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy
2026-02-20T22:40:52.872 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph health --format=json
2026-02-20T22:40:52.987 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2834727457' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/2604038117' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3104209702' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3044383888' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1410793654' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3620011095' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3868616531' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/1960767519' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:53.011 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:52 trial160 ceph-mon[35528]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2834727457' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/2604038117' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3104209702' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3044383888' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1410793654' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3620011095' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3868616531' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/1960767519' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:53.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:52 trial192 ceph-mon[41689]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:53.224 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:53.224 INFO:teuthology.orchestra.run.trial160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]}
2026-02-20T22:40:53.262 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done
2026-02-20T22:40:53.262 INFO:tasks.cephadm:Setup complete, yielding
2026-02-20T22:40:53.262 INFO:teuthology.run_tasks:Running task cephadm.shell...
2026-02-20T22:40:53.276 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:40:53.276 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch status'
2026-02-20T22:40:53.392 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:53.622 INFO:teuthology.orchestra.run.trial160.stdout:Backend: cephadm
2026-02-20T22:40:53.622 INFO:teuthology.orchestra.run.trial160.stdout:Available: Yes
2026-02-20T22:40:53.623 INFO:teuthology.orchestra.run.trial160.stdout:Paused: No
2026-02-20T22:40:53.660 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch ps'
2026-02-20T22:40:53.776 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:53.890 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:53 trial160 ceph-mon[35528]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:53.890 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:53 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/3759571413' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch
2026-02-20T22:40:53.890 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:53 trial160 ceph-mon[35528]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:NAME                    HOST      PORTS             STATUS         REFRESHED  AGE  MEM USE  MEM LIM  VERSION               IMAGE ID      CONTAINER ID
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:alertmanager.trial160   trial160  *:9093,9094       running (32s)    10s ago  58s    22.8M        -  0.28.1                91c01b3cec9b  4147cbd0ddde
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter.trial160  trial160  *:9926            running (61s)    10s ago  61s    8720k        -  20.2.0-677-gf7870454  191e36ff62c2  11e3a7fb87a9
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter.trial192  trial192  *:9926            running (43s)    10s ago  43s    9114k        -  20.2.0-677-gf7870454  191e36ff62c2  716b9a60d897
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:crash.trial160          trial160                    running (61s)    10s ago  61s    7646k        -  20.2.0-677-gf7870454  191e36ff62c2  2819b3c898bd
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:crash.trial192          trial192                    running (42s)    10s ago  42s    7629k        -  20.2.0-677-gf7870454  191e36ff62c2  caf83b49781a
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:grafana.trial160        trial160  *:3000            running (31s)    10s ago  53s     116M        -  12.3.1                5cdab57891ea  a02c96255c61
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:mgr.trial160.pdmeos     trial160  *:9283,8765,8443  running (87s)    10s ago  87s     547M        -  20.2.0-677-gf7870454  191e36ff62c2  723cbab24b32
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:mgr.trial192.inkpau     trial192  *:8443,9283,8765  running (40s)    10s ago  40s     492M        -  20.2.0-677-gf7870454  191e36ff62c2  07fbcd9b771b
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:mon.trial160            trial160                    running (88s)    10s ago  89s    51.2M    2048M  20.2.0-677-gf7870454  191e36ff62c2  1879dfeec923
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:mon.trial192            trial192                    running (39s)    10s ago  39s    45.0M    2048M  20.2.0-677-gf7870454  191e36ff62c2  e4209d42630b
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter.trial160  trial160  *:9100            running (59s)    10s ago  59s    12.6M        -  1.9.1                 255ec253085f  d6955b9ab47a
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter.trial192  trial192  *:9100            running (41s)    10s ago  41s    12.1M        -  1.9.1                 255ec253085f  8156434efb96
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.0                   trial192                    running (19s)    10s ago  19s    32.2M    19.9G  20.2.0-677-gf7870454  191e36ff62c2  ca0214ecf615
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.1                   trial160                    running (19s)    10s ago  19s    33.0M    19.2G  20.2.0-677-gf7870454  191e36ff62c2  52196191b196
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.2                   trial192                    running (17s)    10s ago  17s    32.2M    19.9G  20.2.0-677-gf7870454  191e36ff62c2  89718913c9dc
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.3                   trial160                    running (17s)    10s ago  16s    33.3M    19.2G  20.2.0-677-gf7870454  191e36ff62c2  223d5d69a945
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.4                   trial192                    running (15s)    10s ago  15s    32.3M    19.9G  20.2.0-677-gf7870454  191e36ff62c2  bf66c8ae9dc6
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.5                   trial160                    running (14s)    10s ago  14s    32.2M    19.2G  20.2.0-677-gf7870454  191e36ff62c2  c06c443f4e57
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.6                   trial192                    running (12s)    10s ago  12s    30.4M    19.9G  20.2.0-677-gf7870454  191e36ff62c2  136b3db90ab5
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:osd.7                   trial160                    running (12s)    10s ago  12s    28.4M    19.2G  20.2.0-677-gf7870454  191e36ff62c2  a3fb9b70c7a6
2026-02-20T22:40:53.999 INFO:teuthology.orchestra.run.trial160.stdout:prometheus.trial160     trial160  *:9095            running (30s)    10s ago  51s    38.7M        -  3.6.0                 4fcecf061b74  517d82595fde
2026-02-20T22:40:54.054 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch ls'
2026-02-20T22:40:54.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:53 trial192 ceph-mon[41689]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:54.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:53 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/3759571413' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch
2026-02-20T22:40:54.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:53 trial192 ceph-mon[41689]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:54.169 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:NAME                       PORTS        RUNNING  REFRESHED  AGE  PLACEMENT
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:alertmanager               ?:9093,9094      1/1  10s ago    72s  count:1
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter              ?:9926           2/2  10s ago    73s  *
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:crash                                       2/2  10s ago    74s  *
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:grafana                    ?:3000           1/1  10s ago    73s  count:1
2026-02-20T22:40:54.383 INFO:teuthology.orchestra.run.trial160.stdout:mgr                                         2/2  10s ago    74s  count:2
2026-02-20T22:40:54.384 INFO:teuthology.orchestra.run.trial160.stdout:mon                                         2/2  10s ago    59s  trial160:10.20.193.160=trial160;trial192:10.20.193.192=trial192;count:2
2026-02-20T22:40:54.384 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter              ?:9100           2/2  10s ago    73s  *
2026-02-20T22:40:54.384 INFO:teuthology.orchestra.run.trial160.stdout:osd.all-available-devices                     8  10s ago    31s  *
2026-02-20T22:40:54.384 INFO:teuthology.orchestra.run.trial160.stdout:prometheus                 ?:9095           1/1  10s ago    73s  count:1
2026-02-20T22:40:54.421 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch host ls'
2026-02-20T22:40:54.536 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:54.753 INFO:teuthology.orchestra.run.trial160.stdout:HOST      ADDR           LABELS  STATUS
2026-02-20T22:40:54.753 INFO:teuthology.orchestra.run.trial160.stdout:trial160  10.20.193.160
2026-02-20T22:40:54.753 INFO:teuthology.orchestra.run.trial160.stdout:trial192  10.20.193.192
2026-02-20T22:40:54.753 INFO:teuthology.orchestra.run.trial160.stdout:2 hosts in cluster
2026-02-20T22:40:54.790 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch device ls'
2026-02-20T22:40:54.910 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:55.025 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:54 trial160 ceph-mon[35528]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.025 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:54 trial160 ceph-mon[35528]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.025 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:54 trial160 ceph-mon[35528]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:55.025 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:54 trial160 ceph-mon[35528]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:54 trial192 ceph-mon[41689]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:54 trial192 ceph-mon[41689]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:54 trial192 ceph-mon[41689]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:55.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:54 trial192 ceph-mon[41689]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:55.129 INFO:teuthology.orchestra.run.trial160.stdout:HOST      PATH          TYPE  DEVICE ID                              SIZE  AVAILABLE  REFRESHED  REJECT REASONS
2026-02-20T22:40:55.129 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme1n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_092510AB69B9  1788G  No         10s ago    Has a FileSystem, LVM detected
2026-02-20T22:40:55.129 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme2n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_092510AB699B  1788G  No         10s ago    Has a FileSystem, LVM detected
2026-02-20T22:40:55.129 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme3n1  ssd   Linux_861ea7682dabd119f7ac             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.129 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme4n1  ssd   Linux_626ccc8886957ac31971             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme5n1  ssd   Linux_581b2612f39fc9008265             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme6n1  ssd   Linux_9a30a09e2d9f7bf2c46a             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme1n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_012510B00295  1788G  No         10s ago    Has a FileSystem, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme2n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_012510B002AA  1788G  No         10s ago    Has a FileSystem, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme3n1  ssd   Linux_9224ef463e42b9247c66             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme4n1  ssd   Linux_da8942577159d4ebd296             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme5n1  ssd   Linux_48bbdeb254abff1a7b96             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.130 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme6n1  ssd   Linux_f0d505abb275dbf36d23             700G  No         10s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:40:55.168 INFO:teuthology.run_tasks:Running task cephadm.shell...
2026-02-20T22:40:55.190 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:40:55.190 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch apply rbd-mirror "--placement=*"'
2026-02-20T22:40:55.306 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:55.534 INFO:teuthology.orchestra.run.trial160.stdout:Scheduled rbd-mirror update...
2026-02-20T22:40:55.589 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch apply cephfs-mirror "--placement=*"'
2026-02-20T22:40:55.710 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:55.950 INFO:teuthology.orchestra.run.trial160.stdout:Scheduled cephfs-mirror update...
2026-02-20T22:40:55.994 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service...
2026-02-20T22:40:56.011 INFO:tasks.cephadm:Waiting for ceph service rbd-mirror to start (timeout 300)...
2026-02-20T22:40:56.011 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:40:56.132 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:56.371 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:56.371 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:40:43.818355Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:10.763137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:40:43.818250Z", "ports": [9926], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:55.946968Z service:cephfs-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "running": 0, "size": 2}}, {"events": ["2026-02-20T22:40:11.443860Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:40:43.818286Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:40:43.818402Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:13.300896Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:40:43.818211Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:19.142503Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:40:43.818152Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:12.582341Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:40:43.818322Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:23.228672Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:40:43.818474Z", "running": 8, "size": 8}}, {"events": ["2026-02-20T22:40:19.145063Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:40:43.818439Z", "ports": [9095], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:55.531595Z service:rbd-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "running": 0, "size": 2}}]
2026-02-20T22:40:56.413 INFO:tasks.cephadm:rbd-mirror has 0/2
2026-02-20T22:40:56.817 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "rbd-mirror", "placement": "*", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: Saving service rbd-mirror spec with placement *
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.818 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial192.rupgeq", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]} : dispatch
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial192.rupgeq", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]}]': finished
2026-02-20T22:40:56.819 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:56 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:56.837 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "rbd-mirror", "placement": "*", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: Saving service rbd-mirror spec with placement *
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial192.rupgeq", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]} : dispatch
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial192.rupgeq", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]}]': finished
2026-02-20T22:40:56.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:56 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:57.414 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:40:57.536 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "cephfs-mirror", "placement": "*", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: Saving service cephfs-mirror spec with placement *
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: Deploying daemon rbd-mirror.trial192.rupgeq on trial192
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:57.678 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial160.kxqwbm", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]} : dispatch
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial160.kxqwbm", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]}]': finished
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:57.679 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:57 trial160 ceph-mon[35528]: from='client.? 10.20.193.192:0/4022971763' entity='client.rbd-mirror.trial192.rupgeq' cmd={"prefix": "config-key get", "key": "rbd/mirror/site_name"} : dispatch
2026-02-20T22:40:57.811 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:57.812 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:40:56.498877Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:10.763137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:40:56.150794Z", "ports": [9926], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:55.946968Z service:cephfs-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "running": 0, "size": 2}}, {"events": ["2026-02-20T22:40:11.443860Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:40:56.150930Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:40:56.498933Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:13.300896Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:40:56.151021Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:19.142503Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:40:56.151052Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:12.582341Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:40:56.150988Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:23.228672Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:40:56.151082Z", "running": 8, "size": 8}}, {"events": ["2026-02-20T22:40:19.145063Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:40:56.498978Z", "ports": [9095], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:57.216470Z service:rbd-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "running": 0, "size": 2}}]
2026-02-20T22:40:57.851 INFO:tasks.cephadm:rbd-mirror has 0/2
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "cephfs-mirror", "placement": "*", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: Saving service cephfs-mirror spec with placement *
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: Deploying daemon rbd-mirror.trial192.rupgeq on trial192
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:40:57.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial160.kxqwbm", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]} : dispatch
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.rbd-mirror.trial160.kxqwbm", "caps": ["mon", "profile rbd-mirror", "osd", "profile rbd"]}]': finished
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:57.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:57 trial192 ceph-mon[41689]: from='client.? 10.20.193.192:0/4022971763' entity='client.rbd-mirror.trial192.rupgeq' cmd={"prefix": "config-key get", "key": "rbd/mirror/site_name"} : dispatch
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: Deploying daemon rbd-mirror.trial160.kxqwbm on trial160
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='client.? 10.20.193.160:0/237003485' entity='client.rbd-mirror.trial160.kxqwbm' cmd={"prefix": "config-key get", "key": "rbd/mirror/site_name"} : dispatch
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial192.vesyek", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]} : dispatch
2026-02-20T22:40:58.838 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial192.vesyek", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]}]': finished
2026-02-20T22:40:58.839 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:58 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:58.852 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:40:58.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: Deploying daemon rbd-mirror.trial160.kxqwbm on trial160
2026-02-20T22:40:58.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='client.14632 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:40:58.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='client.? 10.20.193.160:0/237003485' entity='client.rbd-mirror.trial160.kxqwbm' cmd={"prefix": "config-key get", "key": "rbd/mirror/site_name"} : dispatch
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial192.vesyek", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]} : dispatch
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial192.vesyek", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]}]': finished
2026-02-20T22:40:58.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:58 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:40:58.975 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:40:59.283 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:40:59.283 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:40:56.498877Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:10.763137Z service:ceph-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:40:56.150794Z", "ports": [9926], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:58.672782Z service:cephfs-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "running": 0, "size": 2}}, {"events": ["2026-02-20T22:40:11.443860Z service:crash [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:40:56.150930Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:40:56.498933Z", "ports": [3000], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:13.300896Z service:mgr [INFO] \"service was created\""], "placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:40:56.151021Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:19.142503Z service:mon [INFO] \"service was created\""], "placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:40:56.151052Z", "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:12.582341Z service:node-exporter [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:40:56.150988Z", "ports": [9100], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:40:23.228672Z service:osd.all-available-devices [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:40:56.151082Z", "running": 8, "size": 8}}, {"events": ["2026-02-20T22:40:19.145063Z service:prometheus [INFO] \"service was created\""], "placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:40:56.498978Z", "ports": [9095], "running": 1, "size": 1}}, {"events": ["2026-02-20T22:40:57.936770Z service:rbd-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "running": 0, "size": 2}}]
2026-02-20T22:40:59.324 INFO:tasks.cephadm:rbd-mirror has 0/2
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: Deploying daemon cephfs-mirror.trial192.vesyek on trial192
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial160.hzgsqx", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]} : dispatch
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial160.hzgsqx", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]}]': finished
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: Deploying daemon cephfs-mirror.trial160.hzgsqx on trial160
2026-02-20T22:41:00.088 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.089 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:40:59 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr module enable", "module": "mirroring"} : dispatch
2026-02-20T22:41:00.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: Deploying daemon cephfs-mirror.trial192.vesyek on trial192
2026-02-20T22:41:00.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:00.117 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial160.hzgsqx", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]} : dispatch
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "auth get-or-create", "entity": "client.cephfs-mirror.trial160.hzgsqx", "caps": ["mon", "profile cephfs-mirror", "mds", "allow r", "osd", "allow rw tag cephfs metadata=*, allow r tag cephfs data=*", "mgr", "allow r"]}]': finished
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: Deploying daemon cephfs-mirror.trial160.hzgsqx on trial160
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:00.118 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:40:59 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr module enable", "module": "mirroring"} : dispatch
2026-02-20T22:41:00.325 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:41:00.451 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:00.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:00 trial192 ceph-mon[41689]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:00.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:00 trial192 ceph-mon[41689]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "mgr module enable", "module": "mirroring"}]': finished
2026-02-20T22:41:00.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:00 trial192 ceph-mon[41689]: mgrmap e20: trial160.pdmeos(active, since 53s), standbys: trial192.inkpau
2026-02-20T22:41:01.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:00 trial160 ceph-mon[35528]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:01.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:00 trial160 ceph-mon[35528]: from='mgr.14213 10.20.193.160:0/975027583' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "mgr module enable", "module": "mirroring"}]': finished
2026-02-20T22:41:01.087 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:00 trial160 ceph-mon[35528]: mgrmap e20: trial160.pdmeos(active, since 53s), standbys: trial192.inkpau
2026-02-20T22:41:03.433 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: Standby manager daemon trial192.inkpau restarted
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: Standby manager daemon trial192.inkpau started
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/crt"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/key"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: Active manager daemon trial160.pdmeos restarted
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: Activating manager daemon trial160.pdmeos
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: osdmap e26: 8 total, 8 up, 8 in
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: mgrmap e21: trial160.pdmeos(active, starting, since 0.0059188s), standbys: trial192.inkpau
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:41:03.434 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial192.inkpau", "id": "trial192.inkpau"} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:41:03.435 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:03 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:41:03.479 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: Standby manager daemon trial192.inkpau restarted
2026-02-20T22:41:03.479 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: Standby manager daemon trial192.inkpau started
2026-02-20T22:41:03.479 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/crt"} : dispatch
2026-02-20T22:41:03.479 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch
2026-02-20T22:41:03.479 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/trial192.inkpau/key"} : dispatch
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.? 10.20.193.192:0/3215726242' entity='mgr.trial192.inkpau' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: Active manager daemon trial160.pdmeos restarted
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: Activating manager daemon trial160.pdmeos
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: osdmap e26: 8 total, 8 up, 8 in
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: mgrmap e21: trial160.pdmeos(active, starting, since 0.0059188s), standbys: trial192.inkpau
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial160"} : dispatch
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata", "id": "trial192"} : dispatch
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial160.pdmeos", "id": "trial160.pdmeos"} : dispatch
2026-02-20T22:41:03.480 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr metadata", "who": "trial192.inkpau", "id": "trial192.inkpau"} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 0} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 1} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 2} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 3} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 4} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 5} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 6} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata", "id": 7} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mds metadata"} : dispatch
2026-02-20T22:41:03.481 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd metadata"} : dispatch
2026-02-20T22:41:03.482 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:03 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mon metadata"} : dispatch
2026-02-20T22:41:04.238 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:41:04.238 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:40:56.498877Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:40:56.498742Z", "ports": [9926], "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "last_refresh": "2026-02-20T22:41:04.212484Z", "running": 1, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:40:56.498786Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:40:56.498933Z", "ports": [3000], "running": 1, "size": 1}}, {"placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:40:56.498697Z", "running": 2, "size": 2}}, {"placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:40:56.498630Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:40:56.498835Z", "ports": [9100], "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:40:56.499021Z", "running": 8, "size": 8}}, {"placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:40:56.498978Z", "ports": [9095], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "last_refresh": "2026-02-20T22:41:04.212457Z", "running": 1, "size": 2}}]
2026-02-20T22:41:04.279 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:41:04.279 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:41:04.279 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:41:04.279 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:41:04.279 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:41:04.280 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:41:04.280 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:04.280 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:04 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:04.281 INFO:tasks.cephadm:rbd-mirror has 1/2
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: Manager daemon trial160.pdmeos is now available
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "orch get-security-config"} : dispatch
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/mirror_snapshot_schedule"} : dispatch
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/trial160.pdmeos/trash_purge_schedule"} : dispatch
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:04.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:04 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.282 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:41:05.401 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:05.514 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: [20/Feb/2026:22:41:03] ENGINE Bus STARTING
2026-02-20T22:41:05.514 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: [20/Feb/2026:22:41:04] ENGINE Serving on http://10.20.193.160:8765
2026-02-20T22:41:05.514 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: [20/Feb/2026:22:41:04] ENGINE Serving on https://10.20.193.160:7150
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: [20/Feb/2026:22:41:04] ENGINE Bus STARTED
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: [20/Feb/2026:22:41:04] ENGINE Client ('10.20.193.160', 57452) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)')
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: mgrmap e22: trial160.pdmeos(active, since 1.01956s), standbys: trial192.inkpau
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.515 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:05.516 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:05 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:41:05.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: [20/Feb/2026:22:41:03] ENGINE Bus STARTING
2026-02-20T22:41:05.617 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: [20/Feb/2026:22:41:04] ENGINE Serving on http://10.20.193.160:8765
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: [20/Feb/2026:22:41:04] ENGINE Serving on https://10.20.193.160:7150
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: [20/Feb/2026:22:41:04] ENGINE Bus STARTED
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: [20/Feb/2026:22:41:04] ENGINE Client ('10.20.193.160', 57452) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)')
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: mgrmap e22: trial160.pdmeos(active, since 1.01956s), standbys: trial192.inkpau
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch get-security-config"}]: dispatch
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.618 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:05.619 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:05 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:41:05.630 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:41:05.630 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:41:04.542715Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:41:04.212069Z", "ports": [9926], "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "last_refresh": "2026-02-20T22:41:04.212484Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:41:04.212161Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:41:04.542756Z", "ports": [3000], "running": 1, "size": 1}}, {"placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:41:04.212264Z", "running": 2, "size": 2}}, {"placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:41:04.212305Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:41:04.212215Z", "ports": [9100], "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:41:04.212341Z", "running": 8, "size": 8}}, {"placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:41:04.542790Z", "ports": [9095], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "last_refresh": "2026-02-20T22:41:04.212457Z", "running": 2, "size": 2}}]
2026-02-20T22:41:05.667 INFO:tasks.cephadm:rbd-mirror has 2/2
2026-02-20T22:41:05.667 INFO:teuthology.run_tasks:Running task cephadm.wait_for_service...
2026-02-20T22:41:05.684 INFO:tasks.cephadm:Waiting for ceph service cephfs-mirror to start (timeout 300)...
2026-02-20T22:41:05.684 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph orch ls -f json
2026-02-20T22:41:05.802 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:06.024 INFO:teuthology.orchestra.run.trial160.stdout:
2026-02-20T22:41:06.024 INFO:teuthology.orchestra.run.trial160.stdout:[{"placement": {"count": 1}, "service_name": "alertmanager", "service_type": "alertmanager", "status": {"created": "2026-02-20T22:39:41.545932Z", "last_refresh": "2026-02-20T22:41:04.542715Z", "ports": [9093, 9094], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "ceph-exporter", "service_type": "ceph-exporter", "spec": {"prio_limit": 5, "stats_period": 5}, "status": {"created": "2026-02-20T22:39:40.465004Z", "last_refresh": "2026-02-20T22:41:04.212069Z", "ports": [9926], "running": 2, "size": 2}}, {"events": ["2026-02-20T22:41:05.928296Z service:cephfs-mirror [INFO] \"service was created\""], "placement": {"host_pattern": "*"}, "service_name": "cephfs-mirror", "service_type": "cephfs-mirror", "status": {"created": "2026-02-20T22:40:55.944116Z", "last_refresh": "2026-02-20T22:41:04.212484Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "crash", "service_type": "crash", "status": {"created": "2026-02-20T22:39:40.212114Z", "last_refresh": "2026-02-20T22:41:04.212161Z", "running": 2, "size": 2}}, {"placement": {"count": 1}, "service_name": "grafana", "service_type": "grafana", "spec": {"anonymous_access": true, "protocol": "https"}, "status": {"created": "2026-02-20T22:39:41.076521Z", "last_refresh": "2026-02-20T22:41:04.542756Z", "ports": [3000], "running": 1, "size": 1}}, {"placement": {"count": 2}, "service_name": "mgr", "service_type": "mgr", "status": {"created": "2026-02-20T22:39:39.963489Z", "last_refresh": "2026-02-20T22:41:04.212264Z", "running": 2, "size": 2}}, {"placement": {"count": 2, "hosts": ["trial160:10.20.193.160=trial160", "trial192:10.20.193.192=trial192"]}, "service_name": "mon", "service_type": "mon", "status": {"created": "2026-02-20T22:39:55.005485Z", "last_refresh": "2026-02-20T22:41:04.212305Z", "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_name": "node-exporter", "service_type": "node-exporter", "status": {"created": "2026-02-20T22:39:41.311840Z", "last_refresh": "2026-02-20T22:41:04.212215Z", "ports": [9100], "running": 2, "size": 2}}, {"placement": {"host_pattern": "*"}, "service_id": "all-available-devices", "service_name": "osd.all-available-devices", "service_type": "osd", "spec": {"data_devices": {"all": true}, "filter_logic": "AND", "objectstore": "bluestore"}, "status": {"created": "2026-02-20T22:40:23.225809Z", "last_refresh": "2026-02-20T22:41:04.212341Z", "running": 8, "size": 8}}, {"placement": {"count": 1}, "service_name": "prometheus", "service_type": "prometheus", "status": {"created": "2026-02-20T22:39:40.831785Z", "last_refresh": "2026-02-20T22:41:04.542790Z", "ports": [9095], "running": 1, "size": 1}}, {"placement": {"host_pattern": "*"}, "service_name": "rbd-mirror", "service_type": "rbd-mirror", "status": {"created": "2026-02-20T22:40:55.529131Z", "last_refresh": "2026-02-20T22:41:04.212457Z", "running": 2, "size": 2}}]
2026-02-20T22:41:06.065 INFO:tasks.cephadm:cephfs-mirror has 2/2
2026-02-20T22:41:06.065 INFO:teuthology.run_tasks:Running task cephadm.shell...
2026-02-20T22:41:06.081 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:41:06.081 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"''
2026-02-20T22:41:06.203 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:06.265 INFO:teuthology.orchestra.run.trial160.stdout:167 167
2026-02-20T22:41:06.294 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch status'
2026-02-20T22:41:06.410 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:06.637 INFO:teuthology.orchestra.run.trial160.stdout:Backend: cephadm
2026-02-20T22:41:06.637 INFO:teuthology.orchestra.run.trial160.stdout:Available: Yes
2026-02-20T22:41:06.637 INFO:teuthology.orchestra.run.trial160.stdout:Paused: No
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Adjusting osd_memory_target on trial192 to 19955M
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Adjusting osd_memory_target on trial160 to 19187M
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial160:/etc/ceph/ceph.conf
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial192:/etc/ceph/ceph.conf
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial192:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial160:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr module enable", "module": "mirroring"} : dispatch
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "mgr module enable", "module": "mirroring"}]': finished
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: mgrmap e23: trial160.pdmeos(active, since 2s), standbys: trial192.inkpau
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.673 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:06 trial160 ceph-mon[35528]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:41:06.674 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch ps'
2026-02-20T22:41:06.789 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Adjusting osd_memory_target on trial192 to 19955M
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Adjusting osd_memory_target on trial160 to 19187M
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial160:/etc/ceph/ceph.conf
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial192:/etc/ceph/ceph.conf
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.conf
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial192:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial160:/etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: Updating trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/config/ceph.client.admin.keyring
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "mgr module enable", "module": "mirroring"} : dispatch
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd='[{"prefix": "mgr module enable", "module": "mirroring"}]': finished
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: mgrmap e23: trial160.pdmeos(active, since 2s), standbys: trial192.inkpau
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:41:06.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config dump", "format": "json"} : dispatch
2026-02-20T22:41:06.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "config generate-minimal-conf"} : dispatch
2026-02-20T22:41:06.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch
2026-02-20T22:41:06.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos'
2026-02-20T22:41:06.868 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:06 trial192 ceph-mon[41689]: from='mgr.14664 10.20.193.160:0/4069090327' entity='mgr.trial160.pdmeos' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:NAME                           HOST      PORTS             STATUS          REFRESHED   AGE  MEM USE  MEM LIM  VERSION               IMAGE ID      CONTAINER ID
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:alertmanager.trial160          trial160  *:9093,9094       running (45s)      2s ago   71s    22.8M        -  0.28.1                91c01b3cec9b  4147cbd0ddde
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter.trial160         trial160  *:9926            running (74s)      2s ago   74s    9185k        -  20.2.0-677-gf7870454  191e36ff62c2  11e3a7fb87a9
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter.trial192         trial192  *:9926            running (56s)      2s ago   56s    9433k        -  20.2.0-677-gf7870454  191e36ff62c2  716b9a60d897
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:cephfs-mirror.trial160.hzgsqx  trial160                    running (7s)       2s ago    7s    16.1M        -  20.2.0-677-gf7870454  191e36ff62c2  c0097cbc5d3b
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:cephfs-mirror.trial192.vesyek  trial192                    running (8s)       2s ago    8s    16.1M        -  20.2.0-677-gf7870454  191e36ff62c2  b43e04709352
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:crash.trial160                 trial160                    running (74s)      2s ago   74s    7646k        -  20.2.0-677-gf7870454  191e36ff62c2  2819b3c898bd
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:crash.trial192                 trial192                    running (55s)      2s ago   55s    7629k        -  20.2.0-677-gf7870454  191e36ff62c2  caf83b49781a
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:grafana.trial160               trial160  *:3000            running (44s)      2s ago   66s     116M        -  12.3.1                5cdab57891ea  a02c96255c61
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:mgr.trial160.pdmeos            trial160  *:9283,8765,8443  running (100s)     2s ago  100s     512M        -  20.2.0-677-gf7870454  191e36ff62c2  723cbab24b32
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:mgr.trial192.inkpau            trial192  *:8443,9283,8765  running (53s)      2s ago   53s     490M        -  20.2.0-677-gf7870454  191e36ff62c2  07fbcd9b771b
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:mon.trial160                   trial160                    running (101s)     2s ago  102s    56.5M    2048M  20.2.0-677-gf7870454  191e36ff62c2  1879dfeec923
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:mon.trial192                   trial192                    running (52s)      2s ago   52s    48.7M    2048M  20.2.0-677-gf7870454  191e36ff62c2  e4209d42630b
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter.trial160         trial160  *:9100            running (72s)      2s ago   72s    12.7M        -  1.9.1                 255ec253085f  d6955b9ab47a
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter.trial192         trial192  *:9100            running (54s)      2s ago   54s    12.3M        -  1.9.1                 255ec253085f  8156434efb96
2026-02-20T22:41:07.019 INFO:teuthology.orchestra.run.trial160.stdout:osd.0                          trial192                    running (32s)      2s ago   32s    36.1M    19.4G  20.2.0-677-gf7870454  191e36ff62c2  ca0214ecf615
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.1                          trial160                    running (32s)      2s ago   32s    36.5M    18.7G  20.2.0-677-gf7870454  191e36ff62c2  52196191b196
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.2                          trial192                    running (30s)      2s ago   30s    41.8M    19.4G  20.2.0-677-gf7870454  191e36ff62c2  89718913c9dc
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.3                          trial160                    running (30s)      2s ago   29s    37.2M    18.7G  20.2.0-677-gf7870454  191e36ff62c2  223d5d69a945
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.4                          trial192                    running (28s)      2s ago   28s    36.2M    19.4G  20.2.0-677-gf7870454  191e36ff62c2  bf66c8ae9dc6
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.5                          trial160                    running (27s)      2s ago   27s    43.2M    18.7G  20.2.0-677-gf7870454  191e36ff62c2  c06c443f4e57
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.6                          trial192                    running (25s)      2s ago   25s    44.1M    19.4G  20.2.0-677-gf7870454  191e36ff62c2  136b3db90ab5
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:osd.7                          trial160                    running (25s)      2s ago   25s    35.0M    18.7G  20.2.0-677-gf7870454  191e36ff62c2  a3fb9b70c7a6
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:prometheus.trial160            trial160  *:9095            running (43s)      2s ago   64s    45.3M        -  3.6.0                 4fcecf061b74  517d82595fde
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:rbd-mirror.trial160.kxqwbm     trial160                    running (9s)       2s ago    9s    18.6M        -  20.2.0-677-gf7870454  191e36ff62c2  5551d64be436
2026-02-20T22:41:07.020 INFO:teuthology.orchestra.run.trial160.stdout:rbd-mirror.trial192.rupgeq     trial192                    running (9s)       2s ago    9s    17.2M        -  20.2.0-677-gf7870454  191e36ff62c2  56e04dc677d6
2026-02-20T22:41:07.058 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch ls'
2026-02-20T22:41:07.175 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:07.388 INFO:teuthology.orchestra.run.trial160.stdout:NAME                       PORTS        RUNNING  REFRESHED  AGE  PLACEMENT
2026-02-20T22:41:07.388 INFO:teuthology.orchestra.run.trial160.stdout:alertmanager               ?:9093,9094      1/1  2s ago     85s  count:1
2026-02-20T22:41:07.388 INFO:teuthology.orchestra.run.trial160.stdout:ceph-exporter              ?:9926           2/2  3s ago     86s  *
2026-02-20T22:41:07.388 INFO:teuthology.orchestra.run.trial160.stdout:cephfs-mirror                               2/2  3s ago     11s  *
2026-02-20T22:41:07.388 INFO:teuthology.orchestra.run.trial160.stdout:crash                                       2/2  3s ago     87s  *
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:grafana                    ?:3000           1/1  2s ago     86s  count:1
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:mgr                                         2/2  3s ago     87s  count:2
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:mon                                         2/2  3s ago     72s  trial160:10.20.193.160=trial160;trial192:10.20.193.192=trial192;count:2
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:node-exporter              ?:9100           2/2  3s ago     86s  *
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:osd.all-available-devices                     8  3s ago     44s  *
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:prometheus                 ?:9095           1/1  2s ago     86s  count:1
2026-02-20T22:41:07.389 INFO:teuthology.orchestra.run.trial160.stdout:rbd-mirror                                  2/2  3s ago     11s  *
2026-02-20T22:41:07.426 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch host ls'
2026-02-20T22:41:07.541 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:07.564 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:07 trial160 ceph-mon[35528]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:07.564 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:07 trial160 ceph-mon[35528]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:07.564 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:07 trial160 ceph-mon[35528]: Certificate for "grafana_cert (trial160)" is still valid for 1094 days.
2026-02-20T22:41:07.760 INFO:teuthology.orchestra.run.trial160.stdout:HOST      ADDR           LABELS  STATUS
2026-02-20T22:41:07.760 INFO:teuthology.orchestra.run.trial160.stdout:trial160  10.20.193.160
2026-02-20T22:41:07.760 INFO:teuthology.orchestra.run.trial160.stdout:trial192  10.20.193.192
2026-02-20T22:41:07.760 INFO:teuthology.orchestra.run.trial160.stdout:2 hosts in cluster
2026-02-20T22:41:07.797 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch device ls'
2026-02-20T22:41:07.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:07 trial192 ceph-mon[41689]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:07.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:07 trial192 ceph-mon[41689]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch
2026-02-20T22:41:07.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:07 trial192 ceph-mon[41689]: Certificate for "grafana_cert (trial160)" is still valid for 1094 days.
2026-02-20T22:41:07.912 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:HOST      PATH          TYPE  DEVICE ID                              SIZE  AVAILABLE  REFRESHED  REJECT REASONS
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme1n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_092510AB69B9  1788G  No         23s ago    Has a FileSystem, LVM detected
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme2n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_092510AB699B  1788G  No         23s ago    Has a FileSystem, LVM detected
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme3n1  ssd   Linux_861ea7682dabd119f7ac             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme4n1  ssd   Linux_626ccc8886957ac31971             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.127 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme5n1  ssd   Linux_581b2612f39fc9008265             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial160  /dev/nvme6n1  ssd   Linux_9a30a09e2d9f7bf2c46a             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme1n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_012510B00295  1788G  No         23s ago    Has a FileSystem, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme2n1  ssd   MTFDKCC1T9TGP-1BK1JABYY_012510B002AA  1788G  No         23s ago    Has a FileSystem, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme3n1  ssd   Linux_9224ef463e42b9247c66             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme4n1  ssd   Linux_da8942577159d4ebd296             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme5n1  ssd   Linux_48bbdeb254abff1a7b96             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.128 INFO:teuthology.orchestra.run.trial160.stdout:trial192  /dev/nvme6n1  ssd   Linux_f0d505abb275dbf36d23             700G  No         23s ago    Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected
2026-02-20T22:41:08.166 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- bash -c 'ceph orch ls | grep '"'"'^osd.all-available-devices '"'"''
2026-02-20T22:41:08.283 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:08.514 INFO:teuthology.orchestra.run.trial160.stdout:osd.all-available-devices                     8  4s ago     45s  *
2026-02-20T22:41:08.544 DEBUG:teuthology.run_tasks:Unwinding manager cephadm
2026-02-20T22:41:08.557 INFO:tasks.cephadm:Teardown begin
2026-02-20T22:41:08.557 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:08.584 DEBUG:teuthology.orchestra.run.trial192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:41:08.612 INFO:tasks.cephadm:Disabling cephadm mgr module
2026-02-20T22:41:08.612 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 -- ceph mgr module disable cephadm
2026-02-20T22:41:08.737 INFO:teuthology.orchestra.run.trial160.stderr:Inferring config /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/mon.trial160/config
2026-02-20T22:41:08.755 INFO:teuthology.orchestra.run.trial160.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory
2026-02-20T22:41:08.755 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-mon[35528]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.755 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-mon[35528]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.756 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-mon[35528]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:08.756 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-mon[35528]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.756 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-mon[35528]: mgrmap e24: trial160.pdmeos(active, since 4s), standbys: trial192.inkpau
2026-02-20T22:41:08.771 DEBUG:teuthology.orchestra.run:got remote process result: 125
2026-02-20T22:41:08.772 INFO:tasks.cephadm:Cleaning up testdir ceph.* files...
2026-02-20T22:41:08.772 DEBUG:teuthology.orchestra.run.trial160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub
2026-02-20T22:41:08.789 DEBUG:teuthology.orchestra.run.trial192:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub
2026-02-20T22:41:08.806 INFO:tasks.cephadm:Stopping all daemons...
2026-02-20T22:41:08.806 INFO:tasks.cephadm.mon.trial160:Stopping mon.trial160...
2026-02-20T22:41:08.807 DEBUG:teuthology.orchestra.run.trial160:> sudo systemctl stop ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160
2026-02-20T22:41:08.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:08 trial192 ceph-mon[41689]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:08 trial192 ceph-mon[41689]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:08 trial192 ceph-mon[41689]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 5.5 TiB / 5.5 TiB avail
2026-02-20T22:41:08.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:08 trial192 ceph-mon[41689]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch
2026-02-20T22:41:08.867 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:08 trial192 ceph-mon[41689]: mgrmap e24: trial160.pdmeos(active, since 4s), standbys: trial192.inkpau
2026-02-20T22:41:09.046 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 systemd[1]: Stopping Ceph mon.trial160 for fa4d44c0-0eac-11f1-86c6-d404e6e7d460...
2026-02-20T22:41:09.046 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial160[35524]: 2026-02-20T22:41:08.931+0000 7f70060cb640 -1 received  signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.trial160 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false  (PID: 1) UID: 0
2026-02-20T22:41:09.046 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:08 trial160 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial160[35524]: 2026-02-20T22:41:08.931+0000 7f70060cb640 -1 mon.trial160@0(leader) e2 *** Got Signal Terminated ***
2026-02-20T22:41:09.047 INFO:journalctl@ceph.mon.trial160.trial160.stdout:Feb 20 22:41:09 trial160 podman[78573]: 2026-02-20 22:41:09.043847473 +0000 UTC m=+0.124462803 container died 1879dfeec923f96fff4b1aff2781044f93625bdf1f4cbfec09f20138bd162ad6 (image=quay.ceph.io/ceph-ci/ceph:f7870454149f22574086dd2ff55d050585136531, name=ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial160, OSD_FLAVOR=default, ceph=True, CEPH_REF=wip-pdonnell-testing-20260219.182737-tentacle, io.buildah.version=1.33.7, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=f7870454149f22574086dd2ff55d050585136531, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260216, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers@ceph.io>, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git)
2026-02-20T22:41:09.125 DEBUG:teuthology.orchestra.run.trial160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial160.service'
2026-02-20T22:41:09.162 DEBUG:teuthology.orchestra.run:got remote process result: None
2026-02-20T22:41:09.163 INFO:tasks.cephadm.mon.trial160:Stopped mon.trial160
2026-02-20T22:41:09.163 INFO:tasks.cephadm.mon.trial192:Stopping mon.trial192...
2026-02-20T22:41:09.163 DEBUG:teuthology.orchestra.run.trial192:> sudo systemctl stop ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial192
2026-02-20T22:41:09.479 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:09 trial192 systemd[1]: Stopping Ceph mon.trial192 for fa4d44c0-0eac-11f1-86c6-d404e6e7d460...
2026-02-20T22:41:09.480 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:09 trial192 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial192[41685]: 2026-02-20T22:41:09.275+0000 7f32dd357640 -1 received  signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.trial192 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false  (PID: 1) UID: 0
2026-02-20T22:41:09.480 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:09 trial192 ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial192[41685]: 2026-02-20T22:41:09.275+0000 7f32dd357640 -1 mon.trial192@1(peon) e2 *** Got Signal Terminated ***
2026-02-20T22:41:09.480 INFO:journalctl@ceph.mon.trial192.trial192.stdout:Feb 20 22:41:09 trial192 podman[59866]: 2026-02-20 22:41:09.478494726 +0000 UTC m=+0.215874715 container died e4209d42630bd92ddf85971a80980aff48c9a9f14972592aa42c0a8aafe36856 (image=quay.ceph.io/ceph-ci/ceph@sha256:9cc281dd1e60310f8109166bc76e656ce4c5c9b7d9bb9abfc9aa033dfa762ae4, name=ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460-mon-trial192, ceph=True, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.33.7, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team <ceph-maintainers@ceph.io>, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=f7870454149f22574086dd2ff55d050585136531, org.label-schema.build-date=20260216, CEPH_REF=wip-pdonnell-testing-20260219.182737-tentacle, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git)
2026-02-20T22:41:09.565 DEBUG:teuthology.orchestra.run.trial192:> sudo pkill -f 'journalctl -f -n 0 -u ceph-fa4d44c0-0eac-11f1-86c6-d404e6e7d460@mon.trial192.service'
2026-02-20T22:41:09.603 DEBUG:teuthology.orchestra.run:got remote process result: None
2026-02-20T22:41:09.603 INFO:tasks.cephadm.mon.trial192:Stopped mon.trial192
2026-02-20T22:41:09.603 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 --force --keep-logs
2026-02-20T22:41:09.721 INFO:teuthology.orchestra.run.trial160.stdout:Deleting cluster with fsid: fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:41:45.771 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 --force --keep-logs
2026-02-20T22:41:45.892 INFO:teuthology.orchestra.run.trial192.stdout:Deleting cluster with fsid: fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:42:20.994 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:42:21.025 DEBUG:teuthology.orchestra.run.trial192:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring
2026-02-20T22:42:21.053 INFO:tasks.cephadm:Archiving crash dumps...
2026-02-20T22:42:21.055 DEBUG:teuthology.misc:Transferring archived files from trial160:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial160/crash
2026-02-20T22:42:21.056 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash -- .
2026-02-20T22:42:21.093 INFO:teuthology.orchestra.run.trial160.stderr:tar: /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash: Cannot open: No such file or directory
2026-02-20T22:42:21.093 INFO:teuthology.orchestra.run.trial160.stderr:tar: Error is not recoverable: exiting now
2026-02-20T22:42:21.095 DEBUG:teuthology.misc:Transferring archived files from trial192:/var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial192/crash
2026-02-20T22:42:21.096 DEBUG:teuthology.orchestra.run.trial192:> sudo tar c -f - -C /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash -- .
2026-02-20T22:42:21.122 INFO:teuthology.orchestra.run.trial192.stderr:tar: /var/lib/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/crash: Cannot open: No such file or directory
2026-02-20T22:42:21.122 INFO:teuthology.orchestra.run.trial192.stderr:tar: Error is not recoverable: exiting now
2026-02-20T22:42:21.123 INFO:tasks.cephadm:Checking cluster log for badness...
2026-02-20T22:42:21.124 DEBUG:teuthology.orchestra.run.trial160:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_DAEMON_PLACE_FAIL | egrep -v CEPHADM_FAILED_DAEMON | head -n 1
2026-02-20T22:42:21.161 INFO:tasks.cephadm:Compressing logs...
2026-02-20T22:42:21.161 DEBUG:teuthology.orchestra.run.trial160:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-20T22:42:21.203 DEBUG:teuthology.orchestra.run.trial192:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose --
2026-02-20T22:42:21.226 INFO:teuthology.orchestra.run.trial160.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log
2026-02-20T22:42:21.226 INFO:teuthology.orchestra.run.trial160.stderr:: No such file or directory
2026-02-20T22:42:21.226 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial160.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.3.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial160.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial160.kxqwbm.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr: 86.6% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial160.log.gz
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.3.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.5.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial160.kxqwbm.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.7.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr: 47.4% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial160.kxqwbm.log.gz
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.5.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.7.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial160.pdmeos.log
2026-02-20T22:42:21.227 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log
2026-02-20T22:42:21.228 INFO:teuthology.orchestra.run.trial160.stderr: 84.5% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log.gz
2026-02-20T22:42:21.228 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial160.pdmeos.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log
2026-02-20T22:42:21.228 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.1.log
2026-02-20T22:42:21.232 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial160.log
2026-02-20T22:42:21.232 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.1.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log
2026-02-20T22:42:21.232 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial160.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial160.hzgsqx.log
2026-02-20T22:42:21.233 INFO:teuthology.orchestra.run.trial160.stderr: 90.8% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log.gz
2026-02-20T22:42:21.233 INFO:teuthology.orchestra.run.trial160.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log:	/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial160.hzgsqx.log:	 84.0% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log.gz
2026-02-20T22:42:21.233 INFO:teuthology.orchestra.run.trial160.stderr: 49.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial160.hzgsqx.log.gz
2026-02-20T22:42:21.235 INFO:teuthology.orchestra.run.trial192.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory
2026-02-20T22:42:21.235 INFO:teuthology.orchestra.run.trial192.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log
2026-02-20T22:42:21.235 INFO:teuthology.orchestra.run.trial192.stderr:gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.2.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/cephadm.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.0.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.2.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial192.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.0.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial192.rupgeq.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial192.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial192.rupgeq.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.4.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr: 47.5% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.rbd-mirror.trial192.rupgeq.log.gz
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr: 82.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.cephadm.log.gz
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.4.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log
2026-02-20T22:42:21.236 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.6.log
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial192.vesyek.log
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.6.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial192.inkpau.log
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr: 90.9% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.audit.log.gz
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial192.vesyek.log:	gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial192.log
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial192.inkpau.log:	 49.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.cephfs-mirror.trial192.vesyek.log.gz
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:gzip -5 --verbose -- /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial192.log:	 90.0% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-client.ceph-exporter.trial192.log.gz
2026-02-20T22:42:21.237 INFO:teuthology.orchestra.run.trial192.stderr:/var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log:	 82.4% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph.log.gz
2026-02-20T22:42:21.238 INFO:teuthology.orchestra.run.trial192.stderr: 89.9% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial192.inkpau.log.gz
2026-02-20T22:42:21.239 INFO:teuthology.orchestra.run.trial160.stderr: 93.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.3.log.gz
2026-02-20T22:42:21.239 INFO:teuthology.orchestra.run.trial160.stderr: 95.8% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log.gz
2026-02-20T22:42:21.239 INFO:teuthology.orchestra.run.trial160.stderr: 93.3% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.7.log.gz
2026-02-20T22:42:21.239 INFO:teuthology.orchestra.run.trial160.stderr: 93.0% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.1.log.gz
2026-02-20T22:42:21.240 INFO:teuthology.orchestra.run.trial192.stderr: 95.8% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-volume.log.gz
2026-02-20T22:42:21.240 INFO:teuthology.orchestra.run.trial160.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz
2026-02-20T22:42:21.241 INFO:teuthology.orchestra.run.trial192.stderr: 93.1% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.0.log.gz
2026-02-20T22:42:21.242 INFO:teuthology.orchestra.run.trial192.stderr: 93.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.4.log.gz
2026-02-20T22:42:21.243 INFO:teuthology.orchestra.run.trial160.stderr: 94.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.5.log.gz
2026-02-20T22:42:21.249 INFO:teuthology.orchestra.run.trial192.stderr: 94.0% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.2.log.gz
2026-02-20T22:42:21.250 INFO:teuthology.orchestra.run.trial192.stderr: 92.5% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial192.log.gz
2026-02-20T22:42:21.283 INFO:teuthology.orchestra.run.trial192.stderr: 94.2% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-osd.6.log.gz
2026-02-20T22:42:21.284 INFO:teuthology.orchestra.run.trial192.stderr:
2026-02-20T22:42:21.285 INFO:teuthology.orchestra.run.trial192.stderr:real	0m0.064s
2026-02-20T22:42:21.285 INFO:teuthology.orchestra.run.trial192.stderr:user	0m0.124s
2026-02-20T22:42:21.285 INFO:teuthology.orchestra.run.trial192.stderr:sys	0m0.033s
2026-02-20T22:42:21.285 INFO:teuthology.orchestra.run.trial160.stderr: 90.0% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mgr.trial160.pdmeos.log.gz
2026-02-20T22:42:21.309 INFO:teuthology.orchestra.run.trial160.stderr: 91.9% -- replaced with /var/log/ceph/fa4d44c0-0eac-11f1-86c6-d404e6e7d460/ceph-mon.trial160.log.gz
2026-02-20T22:42:21.311 INFO:teuthology.orchestra.run.trial160.stderr:
2026-02-20T22:42:21.311 INFO:teuthology.orchestra.run.trial160.stderr:real	0m0.095s
2026-02-20T22:42:21.311 INFO:teuthology.orchestra.run.trial160.stderr:user	0m0.202s
2026-02-20T22:42:21.312 INFO:teuthology.orchestra.run.trial160.stderr:sys	0m0.041s
2026-02-20T22:42:21.312 INFO:tasks.cephadm:Archiving logs...
2026-02-20T22:42:21.312 DEBUG:teuthology.misc:Transferring archived files from trial160:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial160/log
2026-02-20T22:42:21.314 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /var/log/ceph -- .
2026-02-20T22:42:21.450 DEBUG:teuthology.misc:Transferring archived files from trial192:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial192/log
2026-02-20T22:42:21.451 DEBUG:teuthology.orchestra.run.trial192:> sudo tar c -f - -C /var/log/ceph -- .
2026-02-20T22:42:21.516 INFO:tasks.cephadm:Removing cluster...
2026-02-20T22:42:21.516 DEBUG:teuthology.orchestra.run.trial160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 --force
2026-02-20T22:42:21.635 INFO:teuthology.orchestra.run.trial160.stdout:Deleting cluster with fsid: fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:42:21.692 DEBUG:teuthology.orchestra.run.trial192:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid fa4d44c0-0eac-11f1-86c6-d404e6e7d460 --force
2026-02-20T22:42:21.812 INFO:teuthology.orchestra.run.trial192.stdout:Deleting cluster with fsid: fa4d44c0-0eac-11f1-86c6-d404e6e7d460
2026-02-20T22:42:21.868 INFO:tasks.cephadm:Removing cephadm ...
2026-02-20T22:42:21.868 DEBUG:teuthology.orchestra.run.trial160:> rm -rf /home/ubuntu/cephtest/cephadm
2026-02-20T22:42:21.886 DEBUG:teuthology.orchestra.run.trial192:> rm -rf /home/ubuntu/cephtest/cephadm
2026-02-20T22:42:21.902 INFO:tasks.cephadm:Teardown complete
2026-02-20T22:42:21.902 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop
2026-02-20T22:42:21.913 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial160:/dev/vg_nvme/lv_1...
2026-02-20T22:42:21.913 DEBUG:teuthology.orchestra.run.trial160:> sudo nvme disconnect -n lv_1
2026-02-20T22:42:22.194 INFO:teuthology.orchestra.run.trial160.stdout:NQN:lv_1 disconnected 1 controller(s)
2026-02-20T22:42:22.196 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial160:/dev/vg_nvme/lv_2...
2026-02-20T22:42:22.196 DEBUG:teuthology.orchestra.run.trial160:> sudo nvme disconnect -n lv_2
2026-02-20T22:42:22.446 INFO:teuthology.orchestra.run.trial160.stdout:NQN:lv_2 disconnected 1 controller(s)
2026-02-20T22:42:22.447 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial160:/dev/vg_nvme/lv_3...
2026-02-20T22:42:22.448 DEBUG:teuthology.orchestra.run.trial160:> sudo nvme disconnect -n lv_3
2026-02-20T22:42:22.692 INFO:teuthology.orchestra.run.trial160.stdout:NQN:lv_3 disconnected 1 controller(s)
2026-02-20T22:42:22.694 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial160:/dev/vg_nvme/lv_4...
2026-02-20T22:42:22.695 DEBUG:teuthology.orchestra.run.trial160:> sudo nvme disconnect -n lv_4
2026-02-20T22:42:22.951 INFO:teuthology.orchestra.run.trial160.stdout:NQN:lv_4 disconnected 1 controller(s)
2026-02-20T22:42:22.953 DEBUG:teuthology.orchestra.run.trial160:> set -ex
2026-02-20T22:42:22.953 DEBUG:teuthology.orchestra.run.trial160:> sudo dd of=/scratch_devs
2026-02-20T22:42:22.976 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial192:/dev/vg_nvme/lv_1...
2026-02-20T22:42:22.976 DEBUG:teuthology.orchestra.run.trial192:> sudo nvme disconnect -n lv_1
2026-02-20T22:42:23.230 INFO:teuthology.orchestra.run.trial192.stdout:NQN:lv_1 disconnected 1 controller(s)
2026-02-20T22:42:23.232 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial192:/dev/vg_nvme/lv_2...
2026-02-20T22:42:23.232 DEBUG:teuthology.orchestra.run.trial192:> sudo nvme disconnect -n lv_2
2026-02-20T22:42:23.473 INFO:teuthology.orchestra.run.trial192.stdout:NQN:lv_2 disconnected 1 controller(s)
2026-02-20T22:42:23.474 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial192:/dev/vg_nvme/lv_3...
2026-02-20T22:42:23.475 DEBUG:teuthology.orchestra.run.trial192:> sudo nvme disconnect -n lv_3
2026-02-20T22:42:23.748 INFO:teuthology.orchestra.run.trial192.stdout:NQN:lv_3 disconnected 1 controller(s)
2026-02-20T22:42:23.749 INFO:tasks.nvme_loop:Disconnecting nvme_loop trial192:/dev/vg_nvme/lv_4...
2026-02-20T22:42:23.750 DEBUG:teuthology.orchestra.run.trial192:> sudo nvme disconnect -n lv_4
2026-02-20T22:42:23.997 INFO:teuthology.orchestra.run.trial192.stdout:NQN:lv_4 disconnected 1 controller(s)
2026-02-20T22:42:23.999 DEBUG:teuthology.orchestra.run.trial192:> set -ex
2026-02-20T22:42:23.999 DEBUG:teuthology.orchestra.run.trial192:> sudo dd of=/scratch_devs
2026-02-20T22:42:24.023 DEBUG:teuthology.run_tasks:Unwinding manager clock
2026-02-20T22:42:24.037 INFO:teuthology.task.clock:Checking final clock skew...
2026-02-20T22:42:24.038 DEBUG:teuthology.orchestra.run.trial160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true
2026-02-20T22:42:24.040 DEBUG:teuthology.orchestra.run.trial192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true
2026-02-20T22:42:24.055 INFO:teuthology.orchestra.run.trial160.stderr:bash: line 1: ntpq: command not found
2026-02-20T22:42:24.077 INFO:teuthology.orchestra.run.trial160.stdout:MS Name/IP address         Stratum Poll Reach LastRx Last sample
2026-02-20T22:42:24.077 INFO:teuthology.orchestra.run.trial160.stdout:===============================================================================
2026-02-20T22:42:24.077 INFO:teuthology.orchestra.run.trial160.stdout:^+ 64.44.115.65                  4   6   177     2   +897us[+1029us] +/-   14ms
2026-02-20T22:42:24.077 INFO:teuthology.orchestra.run.trial160.stdout:^- 163-123-153-14.ip4.evals>     4   6    77    64   +485us[ +597us] +/-   50ms
2026-02-20T22:42:24.077 INFO:teuthology.orchestra.run.trial160.stdout:^- 44.190.5.123                  2   6    77    65  +1922us[+2033us] +/-   34ms
2026-02-20T22:42:24.078 INFO:teuthology.orchestra.run.trial160.stdout:^* time.cloudflare.com           3   6   177     1   -379us[ -246us] +/- 6598us
2026-02-20T22:42:24.078 INFO:teuthology.orchestra.run.trial192.stderr:bash: line 1: ntpq: command not found
2026-02-20T22:42:24.081 INFO:teuthology.orchestra.run.trial192.stdout:MS Name/IP address         Stratum Poll Reach LastRx Last sample
2026-02-20T22:42:24.081 INFO:teuthology.orchestra.run.trial192.stdout:===============================================================================
2026-02-20T22:42:24.081 INFO:teuthology.orchestra.run.trial192.stdout:^+ 64.44.115.65                  4   6    77    64  +1015us[+1015us] +/-   14ms
2026-02-20T22:42:24.081 INFO:teuthology.orchestra.run.trial192.stdout:^- 163-123-153-14.ip4.evals>     4   6    77    64   +491us[ +491us] +/-   50ms
2026-02-20T22:42:24.082 INFO:teuthology.orchestra.run.trial192.stdout:^- 44.190.5.123                  2   6    77    64  +1014us[+1014us] +/-   33ms
2026-02-20T22:42:24.082 INFO:teuthology.orchestra.run.trial192.stdout:^* time.cloudflare.com           3   6    77    65   -447us[ -480us] +/- 6636us
2026-02-20T22:42:24.083 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab
2026-02-20T22:42:24.094 INFO:teuthology.task.ansible:Skipping ansible cleanup...
2026-02-20T22:42:24.094 DEBUG:teuthology.run_tasks:Unwinding manager selinux
2026-02-20T22:42:24.108 DEBUG:teuthology.orchestra.run.trial160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log
2026-02-20T22:42:24.177 DEBUG:teuthology.orchestra.run.trial192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log
2026-02-20T22:42:24.235 DEBUG:teuthology.orchestra.run.trial160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0
2026-02-20T22:42:24.263 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:42:24.264 DEBUG:teuthology.orchestra.run.trial192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0
2026-02-20T22:42:24.290 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:42:24.291 DEBUG:teuthology.run_tasks:Unwinding manager pcp
2026-02-20T22:42:24.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer
2026-02-20T22:42:24.325 INFO:teuthology.task.internal:Duration was 368.161013 seconds
2026-02-20T22:42:24.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog
2026-02-20T22:42:24.341 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring...
2026-02-20T22:42:24.341 DEBUG:teuthology.orchestra.run.trial160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart
2026-02-20T22:42:24.344 DEBUG:teuthology.orchestra.run.trial192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart
2026-02-20T22:42:24.380 INFO:teuthology.orchestra.run.trial160.stderr:Redirecting to /bin/systemctl restart rsyslog.service
2026-02-20T22:42:24.380 INFO:teuthology.orchestra.run.trial192.stderr:Redirecting to /bin/systemctl restart rsyslog.service
2026-02-20T22:42:24.651 INFO:teuthology.task.internal.syslog:Checking logs for errors...
2026-02-20T22:42:24.651 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial160.front.sepia.ceph.com
2026-02-20T22:42:24.652 DEBUG:teuthology.orchestra.run.trial160:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1
2026-02-20T22:42:24.678 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial192.front.sepia.ceph.com
2026-02-20T22:42:24.679 DEBUG:teuthology.orchestra.run.trial192:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1
2026-02-20T22:42:24.709 INFO:teuthology.task.internal.syslog:Gathering journactl...
2026-02-20T22:42:24.709 DEBUG:teuthology.orchestra.run.trial160:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log
2026-02-20T22:42:24.720 DEBUG:teuthology.orchestra.run.trial192:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log
2026-02-20T22:42:24.852 INFO:teuthology.task.internal.syslog:Compressing syslogs...
2026-02-20T22:42:24.852 DEBUG:teuthology.orchestra.run.trial160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose --
2026-02-20T22:42:24.854 DEBUG:teuthology.orchestra.run.trial192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose --
2026-02-20T22:42:24.875 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:42:24.875 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:42:24.876 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2026-02-20T22:42:24.877 INFO:teuthology.orchestra.run.trial160.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log
2026-02-20T22:42:24.877 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2026-02-20T22:42:24.878 INFO:teuthology.orchestra.run.trial192.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log
2026-02-20T22:42:24.879 INFO:teuthology.orchestra.run.trial192.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log
2026-02-20T22:42:24.879 INFO:teuthology.orchestra.run.trial192.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log:	gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log
2026-02-20T22:42:24.879 INFO:teuthology.orchestra.run.trial192.stderr:  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz
2026-02-20T22:42:24.879 INFO:teuthology.orchestra.run.trial192.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log:	  0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz
2026-02-20T22:42:24.895 INFO:teuthology.orchestra.run.trial192.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log:	 85.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz
2026-02-20T22:42:24.901 INFO:teuthology.orchestra.run.trial160.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log:	 87.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz
2026-02-20T22:42:24.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo
2026-02-20T22:42:24.913 INFO:teuthology.task.internal:Restoring /etc/sudoers...
2026-02-20T22:42:24.914 DEBUG:teuthology.orchestra.run.trial160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers
2026-02-20T22:42:24.967 DEBUG:teuthology.orchestra.run.trial192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers
2026-02-20T22:42:24.994 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump
2026-02-20T22:42:25.005 DEBUG:teuthology.orchestra.run.trial160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump
2026-02-20T22:42:25.009 DEBUG:teuthology.orchestra.run.trial192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump
2026-02-20T22:42:25.030 INFO:teuthology.orchestra.run.trial160.stdout:kernel.core_pattern = core
2026-02-20T22:42:25.059 INFO:teuthology.orchestra.run.trial192.stdout:kernel.core_pattern = core
2026-02-20T22:42:25.078 DEBUG:teuthology.orchestra.run.trial160:> test -e /home/ubuntu/cephtest/archive/coredump
2026-02-20T22:42:25.100 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:42:25.101 DEBUG:teuthology.orchestra.run.trial192:> test -e /home/ubuntu/cephtest/archive/coredump
2026-02-20T22:42:25.133 DEBUG:teuthology.orchestra.run:got remote process result: 1
2026-02-20T22:42:25.133 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive
2026-02-20T22:42:25.146 INFO:teuthology.task.internal:Transferring archived files...
2026-02-20T22:42:25.147 DEBUG:teuthology.misc:Transferring archived files from trial160:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial160
2026-02-20T22:42:25.147 DEBUG:teuthology.orchestra.run.trial160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2026-02-20T22:42:25.177 DEBUG:teuthology.misc:Transferring archived files from trial192:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-20_16:19:15-orch-wip-pdonnell-testing-20260219.182737-tentacle-distro-default-trial/61616/remote/trial192
2026-02-20T22:42:25.178 DEBUG:teuthology.orchestra.run.trial192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- .
2026-02-20T22:42:25.207 INFO:teuthology.task.internal:Removing archive directory...
2026-02-20T22:42:25.207 DEBUG:teuthology.orchestra.run.trial160:> rm -rf -- /home/ubuntu/cephtest/archive
2026-02-20T22:42:25.215 DEBUG:teuthology.orchestra.run.trial192:> rm -rf -- /home/ubuntu/cephtest/archive
2026-02-20T22:42:25.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload
2026-02-20T22:42:25.275 INFO:teuthology.task.internal:Not uploading archives.
2026-02-20T22:42:25.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.base
2026-02-20T22:42:25.286 INFO:teuthology.task.internal:Tidying up after the test...
2026-02-20T22:42:25.286 DEBUG:teuthology.orchestra.run.trial160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest
2026-02-20T22:42:25.288 DEBUG:teuthology.orchestra.run.trial192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest
2026-02-20T22:42:25.302 INFO:teuthology.orchestra.run.trial160.stdout:   917521      4 drwxr-xr-x   2 ubuntu   ubuntu       4096 Feb 20 22:42 /home/ubuntu/cephtest
2026-02-20T22:42:25.315 INFO:teuthology.orchestra.run.trial192.stdout:   917572      4 drwxr-xr-x   2 ubuntu   ubuntu       4096 Feb 20 22:42 /home/ubuntu/cephtest
2026-02-20T22:42:25.316 DEBUG:teuthology.run_tasks:Unwinding manager kernel
2026-02-20T22:42:25.327 DEBUG:teuthology.run_tasks:Unwinding manager console_log
2026-02-20T22:42:25.361 INFO:teuthology.run:Summary data:
description: orch/cephadm/smoke-roleless/{0-distro/centos_9.stream 0-nvme-loop 1-start
  2-services/mirror 3-final}
duration: 368.1610128879547
owner: scheduled_yuriw@soko04.front.sepia.ceph.com
success: true

2026-02-20T22:42:25.361 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com
2026-02-20T22:42:25.635 INFO:teuthology.run:pass