2024-09-10T19:00:47.736 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T19:00:47.751 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T19:00:47.830 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} email: gabrioux@ibm.com first_in_suite: false job_id: '7900288' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi115.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJt2dnI6bkfKL7dixr6mrtFood1w1GxGQ1oa8OFLvwjs3IzOPkNhv4id+VEWGobMVsi5cGavl+syrPPiXpLoaoc= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8G2QYwENLxG/t5UXlTtA9KH3Jr9AWXfg0ZdcxKYny7NP95z/THKUXKa5qvB6CfWtVSua9cPdMyH5Ie4DD2bZk= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - "set -ex\nfor f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x\n\ do\n echo \"rotating key for $f\"\n K=$(ceph auth get-key $f)\n NK=\"\ $K\"\n ceph orch daemon rotate-key $f\n while [ \"$K\" == \"$NK\" ]; do\n\ \ sleep 5\n NK=$(ceph auth get-key $f)\n done\ndone\n" teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T19:00:47.831 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T19:00:47.831 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T19:00:47.831 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T19:00:47.833 INFO:teuthology.task.internal:Checking packages... 2024-09-10T19:00:47.853 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T19:00:47.854 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T19:00:47.854 INFO:teuthology.packaging:ref: None 2024-09-10T19:00:47.854 INFO:teuthology.packaging:tag: None 2024-09-10T19:00:47.854 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T19:00:47.854 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:00:47.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T19:00:47.994 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T19:00:47.996 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T19:00:48.002 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T19:00:48.002 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T19:00:48.045 INFO:teuthology.task.internal:Saving configuration 2024-09-10T19:00:48.056 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T19:00:48.062 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T19:00:48.088 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:55:43.819427', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJt2dnI6bkfKL7dixr6mrtFood1w1GxGQ1oa8OFLvwjs3IzOPkNhv4id+VEWGobMVsi5cGavl+syrPPiXpLoaoc='} 2024-09-10T19:00:48.111 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:55:43.818224', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK8G2QYwENLxG/t5UXlTtA9KH3Jr9AWXfg0ZdcxKYny7NP95z/THKUXKa5qvB6CfWtVSua9cPdMyH5Ie4DD2bZk='} 2024-09-10T19:00:48.111 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T19:00:48.117 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T19:00:48.117 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T19:00:48.117 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T19:00:48.175 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4fc5197ac0>, signals=[15]) 2024-09-10T19:00:48.175 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T19:00:48.182 INFO:teuthology.task.internal:Opening connections... 2024-09-10T19:00:48.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T19:00:48.184 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:00:48.272 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T19:00:48.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:00:48.350 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T19:00:48.356 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2024-09-10T19:00:48.373 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2024-09-10T19:00:48.373 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2024-09-10T19:00:48.430 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Stream" 2024-09-10T19:00:48.430 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="9" 2024-09-10T19:00:48.430 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2024-09-10T19:00:48.430 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="9" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el9" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:LOGO="fedora-logo-icon" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T19:00:48.431 INFO:teuthology.orchestra.run.smithi115.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T19:00:48.432 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2024-09-10T19:00:48.450 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2024-09-10T19:00:48.468 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2024-09-10T19:00:48.468 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="CentOS Stream" 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="9" 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:ID="centos" 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="rhel fedora" 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="9" 2024-09-10T19:00:48.525 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el9" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:LOGO="fedora-logo-icon" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://centos.org/" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T19:00:48.526 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T19:00:48.526 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2024-09-10T19:00:48.541 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T19:00:48.550 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T19:00:48.554 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T19:00:48.555 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2024-09-10T19:00:48.557 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2024-09-10T19:00:48.581 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T19:00:48.587 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T19:00:48.587 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2024-09-10T19:00:48.613 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2024-09-10T19:00:48.651 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T19:00:48.703 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T19:00:48.722 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T19:00:48.722 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T19:00:48.722 DEBUG:teuthology.orchestra.run.smithi115:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T19:00:48.723 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T19:00:48.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.738 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T19:00:48.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:00:48.739 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-10T19:00:48.794 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-503.el9.x86_64 2024-09-10T19:00:48.795 INFO:teuthology.task.kernel:Running kernel on smithi115: 5.14.0-503.el9.x86_64 2024-09-10T19:00:48.795 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2024-09-10T19:00:48.798 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-503.el9.x86_64 2024-09-10T19:00:48.798 INFO:teuthology.task.kernel:Running kernel on smithi150: 5.14.0-503.el9.x86_64 2024-09-10T19:00:48.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2024-09-10T19:00:51.146 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-10T19:00:51.499 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.2 MB 00:00 2024-09-10T19:00:56.121 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 5.3 MB/s | 20 MB 00:03 2024-09-10T19:00:56.659 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 20 MB 00:04 2024-09-10T19:01:00.355 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T19:01:02.039 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T19:01:02.876 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-10T19:01:03.060 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-10T19:01:04.402 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-10T19:01:05.356 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T19:01:08.482 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T19:01:10.505 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T19:01:10.505 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T19:01:10.587 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:01:10.591 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:01:10.592 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T19:01:10.593 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:01:10.593 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 79 M 2024-09-10T19:01:10.593 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 126 M 2024-09-10T19:01:10.593 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:01:10.812 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-10T19:01:10.817 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 222 kB/s | 23 kB 00:00 2024-09-10T19:01:11.292 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-10T19:01:11.584 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 37 MB/s | 29 MB 00:00 2024-09-10T19:01:11.959 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-10T19:01:11.959 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:01:11.960 INFO:teuthology.orchestra.run.smithi115.stdout:Total 58 MB/s | 79 MB 00:01 2024-09-10T19:01:12.640 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:01:12.802 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T19:01:12.802 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T19:01:12.890 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:01:12.898 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:01:12.898 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T19:01:12.898 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:01:12.898 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:01:12.899 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-10T19:01:12.900 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:01:12.900 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 79 M 2024-09-10T19:01:12.900 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 126 M 2024-09-10T19:01:12.900 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:01:12.978 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:01:12.978 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:01:13.297 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 192 kB/s | 23 kB 00:00 2024-09-10T19:01:13.873 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-10T19:01:13.949 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:01:13.949 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:01:14.381 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:01 2024-09-10T19:01:15.297 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:01:15.298 INFO:teuthology.orchestra.run.smithi150.stdout:Total 33 MB/s | 79 MB 00:02 2024-09-10T19:01:15.978 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:01:16.353 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:01:16.354 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:01:16.391 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:01:17.032 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T19:01:17.126 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:01:17.305 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:01:17.305 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:01:19.766 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:01:19.912 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:01:20.038 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:01:20.416 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T19:01:20.498 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:01:23.299 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:01:23.425 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:01:24.853 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:01:24.887 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:01:28.277 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:01:28.319 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:01:29.755 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:01:33.188 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:07.725 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:07.764 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T19:02:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:02:08.129 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.005 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.006 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.006 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:02:09.006 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:02:09.322 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T19:02:09.402 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.403 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T19:02:09.403 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T19:02:09.403 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T19:02:09.403 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T19:02:09.403 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T19:02:09.403 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T19:02:09.403 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T19:02:09.461 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:09.461 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T19:02:09.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2024-09-10T19:02:09.492 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.06 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 82.el9 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Group : Unspecified 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 8274670 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T19:02:09.493 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : builder@centos.org 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:02:09.494 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T19:02:09.495 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T19:02:09.495 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 5.14.0-505.el9.x86_64 2024-09-10T19:02:09.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T19:02:10.142 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2024-09-10T19:02:11.455 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:11.489 INFO:teuthology.orchestra.run.smithi115.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T19:02:11.495 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:11.508 INFO:teuthology.orchestra.run.smithi115.stderr:done 2024-09-10T19:02:11.515 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2024-09-10T19:02:11.530 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.J0svGzdlJh 2024-09-10T19:02:11.530 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.J0svGzdlJh 2024-09-10T19:02:11.598 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.J0svGzdlJh 2024-09-10T19:02:11.731 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.J0svGzdlJh is 6KB 2024-09-10T19:02:11.752 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.J0svGzdlJh 2024-09-10T19:02:11.767 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T19:02:11.799 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:11.799 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T19:02:11.799 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T19:02:11.799 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T19:02:11.829 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T19:02:11.829 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T19:02:11.829 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T19:02:11.829 INFO:teuthology.orchestra.run.smithi115.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T19:02:11.831 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T19:02:11.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T19:02:12.593 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T19:02:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T19:02:12.594 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:02:12.594 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:02:12.956 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T19:02:13.044 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:13.045 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T19:02:13.045 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T19:02:13.045 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T19:02:13.045 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T19:02:13.045 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T19:02:13.045 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T19:02:13.045 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T19:02:13.104 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T19:02:13.104 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T19:02:13.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.06 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 82.el9 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Group : Unspecified 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 8274670 2024-09-10T19:02:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : builder@centos.org 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : CentOS 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T19:02:13.137 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T19:02:13.138 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T19:02:13.138 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2024-09-10T19:02:13.138 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:02:13.138 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T19:02:13.139 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T19:02:13.139 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 5.14.0-505.el9.x86_64 2024-09-10T19:02:13.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T19:02:13.792 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2024-09-10T19:02:15.217 INFO:teuthology.orchestra.run.smithi150.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T19:02:15.249 INFO:teuthology.orchestra.run.smithi150.stderr:done 2024-09-10T19:02:15.257 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2024-09-10T19:02:15.313 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.5HvxMUq9CS 2024-09-10T19:02:15.313 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5HvxMUq9CS 2024-09-10T19:02:15.382 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.5HvxMUq9CS 2024-09-10T19:02:15.520 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.5HvxMUq9CS is 6KB 2024-09-10T19:02:15.567 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.5HvxMUq9CS 2024-09-10T19:02:15.584 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T19:02:15.644 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T19:02:15.644 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T19:02:15.644 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T19:02:15.645 INFO:teuthology.orchestra.run.smithi150.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T19:02:15.646 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T19:02:15.804 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2024-09-10T19:02:42.013 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T19:02:42.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:02:42.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:02:45.812 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T19:02:45.812 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:02:45.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:03:16.541 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-10T19:03:20.381 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-10T19:03:25.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:03:25.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:03:29.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:03:29.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:03:43.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.115 2024-09-10T19:03:47.773 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2024-09-10T19:03:55.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:03:55.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:03:56.788 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T19:03:57.196 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:03:57.196 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T19:03:57.196 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2024-09-10T19:03:57.213 INFO:teuthology.orchestra.run.smithi115.stdout:5.14.0-505.el9.x86_64 2024-09-10T19:03:57.213 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T19:03:57.213 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T19:03:57.213 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T19:03:58.214 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T19:03:58.214 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T19:03:58.280 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2024-09-10T19:03:58.300 DEBUG:teuthology.parallel:result is None 2024-09-10T19:03:59.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:03:59.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:04:00.211 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-10T19:04:00.607 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:04:00.607 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T19:04:00.608 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2024-09-10T19:04:00.622 INFO:teuthology.orchestra.run.smithi150.stdout:5.14.0-505.el9.x86_64 2024-09-10T19:04:00.623 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T19:04:00.623 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T19:04:00.623 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T19:04:01.624 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T19:04:01.624 DEBUG:teuthology.orchestra.run.smithi150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T19:04:01.710 INFO:teuthology.orchestra.run.smithi150.stdout:ttyS1 2024-09-10T19:04:01.731 DEBUG:teuthology.parallel:result is None 2024-09-10T19:04:01.731 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T19:04:01.738 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T19:04:01.738 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T19:04:01.741 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T19:04:01.756 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T19:04:01.773 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T19:04:01.844 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T19:04:01.844 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T19:04:01.847 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T19:04:01.877 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T19:04:01.884 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T19:04:01.884 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T19:04:01.918 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T19:04:01.945 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T19:04:01.955 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T19:04:01.967 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T19:04:01.977 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T19:04:01.979 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T19:04:01.986 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T19:04:01.986 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T19:04:01.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T19:04:02.025 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T19:04:02.034 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T19:04:02.034 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T19:04:02.059 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T19:04:02.084 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T19:04:02.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T19:04:02.205 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T19:04:02.285 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T19:04:02.334 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:04:02.334 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T19:04:02.402 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T19:04:02.446 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T19:04:02.494 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T19:04:02.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T19:04:02.623 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:04:02.623 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T19:04:02.691 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2024-09-10T19:04:02.693 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2024-09-10T19:04:02.735 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:04:02.786 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:04:03.210 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T19:04:03.217 INFO:teuthology.task.internal:Starting timer... 2024-09-10T19:04:03.217 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T19:04:03.280 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T19:04:03.358 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T19:04:03.358 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2024-09-10T19:04:03.438 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: 2024-09-10T19:04:03.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2024-09-10T19:04:03.503 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: 2024-09-10T19:04:03.505 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T19:04:03.505 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2024-09-10T19:04:03.535 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2024-09-10T19:04:03.535 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2024-09-10T19:04:03.562 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2024-09-10T19:04:03.562 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi115.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2024-09-10T19:04:03.562 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:04:03.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.601 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:04:03.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:04:03.633 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T19:04:03.633 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2024-09-10T19:04:03.803 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2024-09-10T19:04:03.851 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T19:04:03.861 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T19:04:03.864 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T19:04:03.987 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T19:04:03.997 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T19:04:03.998 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi115.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T19:07:45.133 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi115.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2024-09-10T19:07:45.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:07:45.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:07:45.215 DEBUG:teuthology.orchestra.run.smithi115:> true 2024-09-10T19:07:45.292 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi115.front.sepia.ceph.com' 2024-09-10T19:07:45.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:07:45.293 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T19:07:45.369 DEBUG:teuthology.orchestra.run.smithi150:> true 2024-09-10T19:07:45.448 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2024-09-10T19:07:45.448 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T19:07:45.461 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T19:07:45.461 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T19:07:45.462 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:07:45.465 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T19:07:45.465 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:07:45.492 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T19:07:45.506 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T19:07:45.518 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T19:07:45.529 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2024-09-10T19:07:45.533 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T19:07:45.540 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2024-09-10T19:07:45.555 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T19:07:45.558 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2024-09-10T19:07:45.569 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T19:07:45.569 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2024-09-10T19:07:45.583 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T19:07:45.600 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T19:07:45.629 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:07:45.633 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:07:45.633 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T19:07:45.634 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.634 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.634 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.634 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.652 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:07:45.655 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:07:45.655 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-10T19:07:45.656 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.656 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.656 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.656 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:07:45.657 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T19:07:45.670 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T19:07:45.670 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T19:07:45.670 DEBUG:teuthology.orchestra.run.smithi115:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T19:07:45.671 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T19:07:45.671 DEBUG:teuthology.orchestra.run.smithi150:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T19:07:46.022 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:07:46.024 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Removing: 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Removing unused dependencies: 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Remove 2 Packages 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Freed space: 5.5 M 2024-09-10T19:07:46.025 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:07:46.029 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:07:46.029 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:07:46.049 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:07:46.055 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:46.055 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T19:07:46.055 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Removing: 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Removing unused dependencies: 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Remove 2 Packages 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Freed space: 5.5 M 2024-09-10T19:07:46.056 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:07:46.060 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:07:46.060 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:07:46.099 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:07:46.100 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:07:46.117 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:07:46.117 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:07:46.235 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:07:46.260 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:07:46.315 INFO:teuthology.orchestra.run.smithi115.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T19:07:46.338 INFO:teuthology.orchestra.run.smithi150.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T19:07:46.384 INFO:teuthology.orchestra.run.smithi115.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T19:07:46.391 INFO:teuthology.orchestra.run.smithi150.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T19:07:46.842 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T19:07:46.842 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T19:07:46.913 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T19:07:46.913 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T19:07:47.092 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T19:07:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:47.093 INFO:teuthology.orchestra.run.smithi150.stdout:Removed: 2024-09-10T19:07:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T19:07:47.093 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:47.093 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:07:47.185 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T19:07:47.185 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:47.185 INFO:teuthology.orchestra.run.smithi115.stdout:Removed: 2024-09-10T19:07:47.186 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T19:07:47.186 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:47.186 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:07:47.949 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 07:07:00 PM UTC. 2024-09-10T19:07:48.013 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 07:07:00 PM UTC. 2024-09-10T19:07:48.084 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:07:48.085 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-09-10T19:07:48.086 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:48.090 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.0 M 2024-09-10T19:07:48.090 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 9.2 M 2024-09-10T19:07:48.090 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:07:48.148 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:07:48.148 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:07:48.149 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-10T19:07:48.150 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:48.155 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.0 M 2024-09-10T19:07:48.155 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 9.2 M 2024-09-10T19:07:48.155 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:07:48.298 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 396 kB/s | 44 kB 00:00 2024-09-10T19:07:48.348 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 653 kB/s | 106 kB 00:00 2024-09-10T19:07:48.373 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 955 kB/s | 72 kB 00:00 2024-09-10T19:07:48.429 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.0 MB/s | 84 kB 00:00 2024-09-10T19:07:48.524 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.6 MB/s | 906 kB 00:00 2024-09-10T19:07:48.532 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 284 kB/s | 44 kB 00:00 2024-09-10T19:07:48.549 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.7 MB/s | 837 kB 00:00 2024-09-10T19:07:48.549 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:07:48.549 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2024-09-10T19:07:48.557 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 582 kB/s | 106 kB 00:00 2024-09-10T19:07:48.616 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-10T19:07:48.623 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:07:48.637 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:07:48.637 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:07:48.641 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 659 kB/s | 72 kB 00:00 2024-09-10T19:07:48.700 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.7 MB/s | 906 kB 00:00 2024-09-10T19:07:48.759 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.8 MB/s | 837 kB 00:00 2024-09-10T19:07:48.759 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:07:48.759 INFO:teuthology.orchestra.run.smithi150.stdout:Total 3.0 MB/s | 2.0 MB 00:00 2024-09-10T19:07:48.833 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:07:48.847 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:07:48.847 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:07:48.924 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:07:48.924 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:07:49.133 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:07:49.134 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:07:49.334 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:07:49.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T19:07:49.452 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T19:07:49.481 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T19:07:49.559 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:07:49.653 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T19:07:49.687 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T19:07:49.690 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T19:07:49.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T19:07:49.728 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T19:07:49.926 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T19:07:49.955 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T19:07:50.178 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T19:07:50.210 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T19:07:50.401 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T19:07:50.435 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T19:07:51.220 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T19:07:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T19:07:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T19:07:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T19:07:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T19:07:51.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T19:07:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T19:07:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T19:07:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T19:07:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T19:07:51.482 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T19:07:51.483 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T19:07:51.536 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:07:51.537 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:07:51.842 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T19:07:51.842 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:07:51.843 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:07:51.880 DEBUG:teuthology.parallel:result is None 2024-09-10T19:07:52.183 DEBUG:teuthology.parallel:result is None 2024-09-10T19:07:52.183 INFO:teuthology.run_tasks:Running task install... 2024-09-10T19:07:52.191 DEBUG:teuthology.task.install:project ceph 2024-09-10T19:07:52.191 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}, 'extra_packages': ['cephadm']} 2024-09-10T19:07:52.191 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T19:07:52.191 INFO:teuthology.task.install:Using flavor: default 2024-09-10T19:07:52.197 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T19:07:52.197 INFO:teuthology.task.install:extra packages: [] 2024-09-10T19:07:52.197 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T19:07:52.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:07:52.200 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T19:07:52.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:07:52.344 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T19:07:52.344 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T19:07:52.355 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T19:07:52.355 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T19:07:52.454 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T19:07:52.454 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:07:52.455 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T19:07:52.464 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T19:07:52.464 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:07:52.464 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T19:07:52.488 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T19:07:52.488 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T19:07:52.492 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T19:07:52.492 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T19:07:52.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T19:07:52.561 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T19:07:52.641 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T19:07:52.656 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T19:07:52.680 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2024-09-10T19:07:52.682 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2024-09-10T19:07:52.708 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2024-09-10T19:07:52.710 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2024-09-10T19:07:52.910 INFO:teuthology.orchestra.run.smithi150.stdout:45 files removed 2024-09-10T19:07:52.937 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw 2024-09-10T19:07:52.941 INFO:teuthology.orchestra.run.smithi115.stdout:45 files removed 2024-09-10T19:07:52.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2024-09-10T19:07:53.489 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 290 kB/s | 77 kB 00:00 2024-09-10T19:07:53.523 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 285 kB/s | 77 kB 00:00 2024-09-10T19:07:53.773 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-10T19:07:53.816 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 43 kB/s | 11 kB 00:00 2024-09-10T19:07:54.048 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-10T19:07:54.075 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 8.0 kB/s | 1.8 kB 00:00 2024-09-10T19:07:54.515 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.2 MB 00:00 2024-09-10T19:07:55.833 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-10T19:07:56.383 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T19:07:57.709 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T19:08:02.235 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T19:08:03.168 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.5 MB 00:02 2024-09-10T19:08:04.435 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T19:08:04.601 INFO:teuthology.orchestra.run.smithi150.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 19 kB 00:00 2024-09-10T19:08:05.722 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T19:08:05.830 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-10T19:08:11.153 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T19:08:11.328 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-10T19:08:13.548 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:08:13.550 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T19:08:13.551 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout:Install 22 Packages 2024-09-10T19:08:13.552 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2024-09-10T19:08:13.553 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 58 M 2024-09-10T19:08:13.553 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:08:13.775 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T19:08:13.777 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T19:08:13.778 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout:Install 22 Packages 2024-09-10T19:08:13.779 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2024-09-10T19:08:13.780 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:13.780 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 58 M 2024-09-10T19:08:13.780 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:08:14.288 INFO:teuthology.orchestra.run.smithi150.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 11 MB/s | 5.0 MB 00:00 2024-09-10T19:08:14.338 INFO:teuthology.orchestra.run.smithi150.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 505 kB/s | 25 kB 00:00 2024-09-10T19:08:14.430 INFO:teuthology.orchestra.run.smithi150.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 6.8 MB/s | 637 kB 00:00 2024-09-10T19:08:14.522 INFO:teuthology.orchestra.run.smithi150.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 4.9 MB/s | 458 kB 00:00 2024-09-10T19:08:14.864 INFO:teuthology.orchestra.run.smithi150.stdout:(5/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 14 MB/s | 4.9 MB 00:00 2024-09-10T19:08:15.073 INFO:teuthology.orchestra.run.smithi150.stdout:(6/24): ceph-common-17.2.7-1606.g5be57395.el9.x 18 MB/s | 22 MB 00:01 2024-09-10T19:08:15.098 INFO:teuthology.orchestra.run.smithi150.stdout:(7/24): python3-ceph-argparse-17.2.7-1606.g5be5 193 kB/s | 45 kB 00:00 2024-09-10T19:08:15.125 INFO:teuthology.orchestra.run.smithi150.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 2.4 MB/s | 120 kB 00:00 2024-09-10T19:08:15.145 INFO:teuthology.orchestra.run.smithi115.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 7.2 MB/s | 5.0 MB 00:00 2024-09-10T19:08:15.148 INFO:teuthology.orchestra.run.smithi150.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.0 MB/s | 156 kB 00:00 2024-09-10T19:08:15.174 INFO:teuthology.orchestra.run.smithi150.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T19:08:15.179 INFO:teuthology.orchestra.run.smithi115.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 762 kB/s | 25 kB 00:00 2024-09-10T19:08:15.199 INFO:teuthology.orchestra.run.smithi150.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T19:08:15.224 INFO:teuthology.orchestra.run.smithi150.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T19:08:15.246 INFO:teuthology.orchestra.run.smithi115.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 9.4 MB/s | 637 kB 00:00 2024-09-10T19:08:15.313 INFO:teuthology.orchestra.run.smithi115.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 6.7 MB/s | 458 kB 00:00 2024-09-10T19:08:15.366 INFO:teuthology.orchestra.run.smithi150.stdout:(13/24): ceph-radosgw-17.2.7-1606.g5be57395.el9 9.5 MB/s | 14 MB 00:01 2024-09-10T19:08:15.433 INFO:teuthology.orchestra.run.smithi150.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 685 kB/s | 160 kB 00:00 2024-09-10T19:08:15.458 INFO:teuthology.orchestra.run.smithi150.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 194 kB/s | 45 kB 00:00 2024-09-10T19:08:15.492 INFO:teuthology.orchestra.run.smithi150.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-09-10T19:08:15.517 INFO:teuthology.orchestra.run.smithi150.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-10T19:08:15.555 INFO:teuthology.orchestra.run.smithi115.stdout:(5/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 20 MB/s | 4.9 MB 00:00 2024-09-10T19:08:15.568 INFO:teuthology.orchestra.run.smithi150.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-10T19:08:15.593 INFO:teuthology.orchestra.run.smithi150.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-09-10T19:08:15.596 INFO:teuthology.orchestra.run.smithi115.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 1.1 MB/s | 45 kB 00:00 2024-09-10T19:08:15.618 INFO:teuthology.orchestra.run.smithi150.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.4 MB/s | 309 kB 00:00 2024-09-10T19:08:15.644 INFO:teuthology.orchestra.run.smithi150.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 914 kB/s | 67 kB 00:00 2024-09-10T19:08:15.719 INFO:teuthology.orchestra.run.smithi150.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T19:08:15.777 INFO:teuthology.orchestra.run.smithi150.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 19 MB/s | 3.1 MB 00:00 2024-09-10T19:08:15.830 INFO:teuthology.orchestra.run.smithi115.stdout:(7/24): ceph-common-17.2.7-1606.g5be57395.el9.x 16 MB/s | 22 MB 00:01 2024-09-10T19:08:15.855 INFO:teuthology.orchestra.run.smithi115.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 465 kB/s | 120 kB 00:00 2024-09-10T19:08:15.872 INFO:teuthology.orchestra.run.smithi115.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.7 MB/s | 156 kB 00:00 2024-09-10T19:08:15.877 INFO:teuthology.orchestra.run.smithi150.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-10T19:08:15.882 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:15.883 INFO:teuthology.orchestra.run.smithi150.stdout:Total 25 MB/s | 58 MB 00:02 2024-09-10T19:08:15.906 INFO:teuthology.orchestra.run.smithi115.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T19:08:15.931 INFO:teuthology.orchestra.run.smithi115.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 4.9 MB/s | 295 kB 00:00 2024-09-10T19:08:15.942 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:08:15.956 INFO:teuthology.orchestra.run.smithi115.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T19:08:15.975 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:08:15.975 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:08:16.040 INFO:teuthology.orchestra.run.smithi115.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 545 kB/s | 45 kB 00:00 2024-09-10T19:08:16.066 INFO:teuthology.orchestra.run.smithi115.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T19:08:16.124 INFO:teuthology.orchestra.run.smithi115.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-10T19:08:16.158 INFO:teuthology.orchestra.run.smithi115.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-10T19:08:16.183 INFO:teuthology.orchestra.run.smithi115.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-09-10T19:08:16.242 INFO:teuthology.orchestra.run.smithi115.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-09-10T19:08:16.267 INFO:teuthology.orchestra.run.smithi115.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-09-10T19:08:16.292 INFO:teuthology.orchestra.run.smithi115.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T19:08:16.351 INFO:teuthology.orchestra.run.smithi115.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T19:08:16.434 INFO:teuthology.orchestra.run.smithi115.stdout:(22/24): librados2-17.2.7-1606.g5be57395.el9.x8 22 MB/s | 3.1 MB 00:00 2024-09-10T19:08:16.445 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:08:16.446 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:08:16.593 INFO:teuthology.orchestra.run.smithi115.stdout:(23/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-10T19:08:17.657 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:08:17.997 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T19:08:18.097 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:18.183 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:18.400 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T19:08:18.440 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:18.526 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:18.565 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T19:08:18.603 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T19:08:19.048 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T19:08:19.076 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T19:08:19.139 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T19:08:19.176 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T19:08:19.224 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T19:08:19.264 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T19:08:19.326 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T19:08:19.354 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T19:08:19.411 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T19:08:19.451 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T19:08:19.488 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T19:08:19.520 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T19:08:19.822 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T19:08:19.877 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T19:08:19.964 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T19:08:20.038 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T19:08:20.069 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T19:08:20.651 INFO:teuthology.orchestra.run.smithi115.stdout:(24/24): ceph-radosgw-17.2.7-1606.g5be57395.el9 2.3 MB/s | 14 MB 00:06 2024-09-10T19:08:20.657 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:20.657 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.5 MB/s | 58 MB 00:06 2024-09-10T19:08:20.723 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:08:20.757 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:08:20.757 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:08:21.243 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:08:21.243 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:08:22.419 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:08:22.481 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:22.524 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:22.750 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T19:08:22.870 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:22.961 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:23.040 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T19:08:23.060 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:23.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:23.110 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T19:08:23.175 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:23.225 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T19:08:23.225 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T19:08:23.225 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T19:08:23.225 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:23.230 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T19:08:23.259 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T19:08:23.278 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:23.702 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T19:08:23.740 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T19:08:23.803 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T19:08:23.832 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T19:08:23.875 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T19:08:23.912 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T19:08:23.975 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T19:08:24.002 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T19:08:24.068 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T19:08:24.118 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T19:08:24.160 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T19:08:24.192 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T19:08:24.488 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T19:08:24.530 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T19:08:24.620 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T19:08:24.694 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T19:08:24.726 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T19:08:27.098 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:27.131 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:27.668 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T19:08:27.701 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T19:08:27.797 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T19:08:27.798 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T19:08:27.798 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T19:08:27.798 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:28.053 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /root 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2024-09-10T19:08:33.041 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:34.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T19:08:34.465 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T19:08:34.465 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:34.465 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T19:08:34.465 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T19:08:34.466 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T19:08:34.466 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:34.467 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T19:08:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T19:08:34.548 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:35.849 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:35.849 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T19:08:35.850 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T19:08:35.851 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T19:08:36.664 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T19:08:36.665 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T19:08:36.666 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.667 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.667 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:36.667 INFO:teuthology.orchestra.run.smithi150.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T19:08:36.667 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:36.667 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:08:37.052 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-test 2024-09-10T19:08:37.632 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:26 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:08:37.765 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T19:08:37.766 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout:Install 4 Packages 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:37.767 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 48 M 2024-09-10T19:08:37.768 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 193 M 2024-09-10T19:08:37.768 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /root 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2024-09-10T19:08:37.875 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:38.093 INFO:teuthology.orchestra.run.smithi150.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 725 kB/s | 163 kB 00:00 2024-09-10T19:08:38.460 INFO:teuthology.orchestra.run.smithi150.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 174 kB/s | 64 kB 00:00 2024-09-10T19:08:38.501 INFO:teuthology.orchestra.run.smithi150.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 480 kB/s | 304 kB 00:00 2024-09-10T19:08:39.211 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T19:08:39.298 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:39.300 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T19:08:39.389 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T19:08:39.389 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:39.394 INFO:teuthology.orchestra.run.smithi150.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-10T19:08:39.396 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:39.396 INFO:teuthology.orchestra.run.smithi150.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-10T19:08:39.424 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:08:39.444 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:08:39.444 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:08:39.812 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:08:39.812 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:08:40.335 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:08:40.428 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T19:08:40.475 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T19:08:40.504 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T19:08:40.541 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T19:08:40.542 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T19:08:40.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T19:08:40.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T19:08:40.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T19:08:40.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T19:08:40.543 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T19:08:41.428 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T19:08:41.428 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.429 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T19:08:41.430 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:41.431 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:08:41.816 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2024-09-10T19:08:42.384 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:31 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:08:42.523 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:08:42.523 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:42.524 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:08:42.525 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:42.525 INFO:teuthology.orchestra.run.smithi115.stdout:Install 4 Packages 2024-09-10T19:08:42.525 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:42.525 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 48 M 2024-09-10T19:08:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 193 M 2024-09-10T19:08:42.526 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:08:43.009 INFO:teuthology.orchestra.run.smithi115.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-10T19:08:43.058 INFO:teuthology.orchestra.run.smithi115.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-09-10T19:08:43.100 INFO:teuthology.orchestra.run.smithi115.stdout:(3/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 549 kB/s | 163 kB 00:00 2024-09-10T19:08:44.434 INFO:teuthology.orchestra.run.smithi115.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 29 MB/s | 48 MB 00:01 2024-09-10T19:08:44.436 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:44.437 INFO:teuthology.orchestra.run.smithi115.stdout:Total 25 MB/s | 48 MB 00:01 2024-09-10T19:08:44.464 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:08:44.483 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:08:44.483 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:08:44.851 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:08:44.852 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:08:44.934 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T19:08:45.013 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T19:08:45.386 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:08:45.490 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T19:08:45.533 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T19:08:45.561 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T19:08:47.241 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T19:08:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T19:08:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T19:08:47.242 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T19:08:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T19:08:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:47.575 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:08:47.575 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T19:08:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T19:08:47.576 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:47.576 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:08:47.872 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph 2024-09-10T19:08:48.438 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:37 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:08:48.574 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T19:08:48.577 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections 2024-09-10T19:08:48.578 INFO:teuthology.orchestra.run.smithi150.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T19:08:48.579 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout:Install 34 Packages 2024-09-10T19:08:48.580 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:48.581 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 27 M 2024-09-10T19:08:48.581 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 96 M 2024-09-10T19:08:48.581 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:08:49.205 INFO:teuthology.orchestra.run.smithi150.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-10T19:08:49.398 INFO:teuthology.orchestra.run.smithi150.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-10T19:08:49.448 INFO:teuthology.orchestra.run.smithi150.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-10T19:08:49.548 INFO:teuthology.orchestra.run.smithi150.stdout:(4/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-10T19:08:49.573 INFO:teuthology.orchestra.run.smithi150.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 1.9 MB/s | 240 kB 00:00 2024-09-10T19:08:49.700 INFO:teuthology.orchestra.run.smithi150.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T19:08:49.757 INFO:teuthology.orchestra.run.smithi150.stdout:(7/34): python3-cryptography-36.0.1-4.el9.x86_6 6.7 MB/s | 1.2 MB 00:00 2024-09-10T19:08:49.782 INFO:teuthology.orchestra.run.smithi150.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-10T19:08:49.950 INFO:teuthology.orchestra.run.smithi150.stdout:(9/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 28 MB/s | 15 MB 00:00 2024-09-10T19:08:49.971 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T19:08:49.975 INFO:teuthology.orchestra.run.smithi150.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 650 kB/s | 125 kB 00:00 2024-09-10T19:08:50.000 INFO:teuthology.orchestra.run.smithi150.stdout:(11/34): python3-pycparser-2.20-6.el9.noarch.rp 557 kB/s | 135 kB 00:00 2024-09-10T19:08:50.043 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T19:08:50.100 INFO:teuthology.orchestra.run.smithi150.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 348 kB/s | 35 kB 00:00 2024-09-10T19:08:50.126 INFO:teuthology.orchestra.run.smithi150.stdout:(13/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.2 MB/s | 215 kB 00:00 2024-09-10T19:08:50.176 INFO:teuthology.orchestra.run.smithi150.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 858 kB/s | 172 kB 00:00 2024-09-10T19:08:50.201 INFO:teuthology.orchestra.run.smithi150.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 768 kB/s | 77 kB 00:00 2024-09-10T19:08:50.226 INFO:teuthology.orchestra.run.smithi150.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-09-10T19:08:50.268 INFO:teuthology.orchestra.run.smithi150.stdout:(17/34): python3-pytz-2021.1-5.el9.noarch.rpm 358 kB/s | 51 kB 00:00 2024-09-10T19:08:50.293 INFO:teuthology.orchestra.run.smithi150.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-10T19:08:50.336 INFO:teuthology.orchestra.run.smithi150.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 309 kB/s | 11 kB 00:00 2024-09-10T19:08:50.361 INFO:teuthology.orchestra.run.smithi150.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-10T19:08:50.386 INFO:teuthology.orchestra.run.smithi150.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 365 kB/s | 18 kB 00:00 2024-09-10T19:08:50.412 INFO:teuthology.orchestra.run.smithi150.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-10T19:08:50.437 INFO:teuthology.orchestra.run.smithi150.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-10T19:08:50.462 INFO:teuthology.orchestra.run.smithi150.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-10T19:08:50.488 INFO:teuthology.orchestra.run.smithi150.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 924 kB/s | 46 kB 00:00 2024-09-10T19:08:50.513 INFO:teuthology.orchestra.run.smithi150.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T19:08:50.538 INFO:teuthology.orchestra.run.smithi150.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-10T19:08:50.555 INFO:teuthology.orchestra.run.smithi150.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 393 kB/s | 16 kB 00:00 2024-09-10T19:08:50.572 INFO:teuthology.orchestra.run.smithi150.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-10T19:08:50.597 INFO:teuthology.orchestra.run.smithi150.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-10T19:08:50.631 INFO:teuthology.orchestra.run.smithi150.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-10T19:08:50.656 INFO:teuthology.orchestra.run.smithi150.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T19:08:50.681 INFO:teuthology.orchestra.run.smithi150.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 400 kB/s | 20 kB 00:00 2024-09-10T19:08:50.707 INFO:teuthology.orchestra.run.smithi150.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-10T19:08:50.711 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:50.711 INFO:teuthology.orchestra.run.smithi150.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-10T19:08:50.844 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:08:50.880 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:08:50.881 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:08:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:08:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:08:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T19:08:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T19:08:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T19:08:52.221 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T19:08:52.488 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T19:08:52.545 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:52.546 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:08:52.551 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T19:08:52.583 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T19:08:52.624 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T19:08:52.715 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T19:08:52.802 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T19:08:52.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2024-09-10T19:08:52.843 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T19:08:52.879 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T19:08:52.909 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T19:08:52.943 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T19:08:52.985 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T19:08:53.034 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T19:08:53.068 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T19:08:53.103 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T19:08:53.151 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T19:08:53.194 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T19:08:53.281 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T19:08:53.398 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:42 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:08:53.428 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T19:08:53.507 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T19:08:53.533 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T19:08:53.536 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T19:08:53.537 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T19:08:53.538 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout:Install 34 Packages 2024-09-10T19:08:53.539 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:08:53.540 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 27 M 2024-09-10T19:08:53.540 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 96 M 2024-09-10T19:08:53.540 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:08:53.569 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T19:08:53.609 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T19:08:53.657 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T19:08:53.720 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T19:08:53.913 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T19:08:53.971 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T19:08:54.048 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T19:08:54.095 INFO:teuthology.orchestra.run.smithi115.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 46 kB/s | 6.5 kB 00:00 2024-09-10T19:08:54.145 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T19:08:54.195 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T19:08:54.254 INFO:teuthology.orchestra.run.smithi115.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.7 MB/s | 1.4 MB 00:00 2024-09-10T19:08:54.296 INFO:teuthology.orchestra.run.smithi115.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 6.1 MB/s | 2.1 MB 00:00 2024-09-10T19:08:54.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T19:08:54.363 INFO:teuthology.orchestra.run.smithi115.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 3.5 MB/s | 240 kB 00:00 2024-09-10T19:08:54.457 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T19:08:54.489 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T19:08:54.515 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T19:08:54.516 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:54.516 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T19:08:54.516 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T19:08:54.516 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T19:08:54.516 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:54.555 INFO:teuthology.orchestra.run.smithi115.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 8.5 MB/s | 3.9 MB 00:00 2024-09-10T19:08:54.780 INFO:teuthology.orchestra.run.smithi115.stdout:(6/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 29 MB/s | 15 MB 00:00 2024-09-10T19:08:55.372 INFO:teuthology.orchestra.run.smithi115.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 251 kB/s | 253 kB 00:01 2024-09-10T19:08:55.472 INFO:teuthology.orchestra.run.smithi115.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 155 kB/s | 106 kB 00:00 2024-09-10T19:08:55.739 INFO:teuthology.orchestra.run.smithi115.stdout:(9/34): python3-cryptography-36.0.1-4.el9.x86_6 1.0 MB/s | 1.2 MB 00:01 2024-09-10T19:08:55.865 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T19:08:55.895 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T19:08:55.896 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:55.896 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T19:08:55.896 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T19:08:55.896 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T19:08:55.896 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:56.189 INFO:teuthology.orchestra.run.smithi115.stdout:(10/34): python3-pycparser-2.20-6.el9.noarch.rp 165 kB/s | 135 kB 00:00 2024-09-10T19:08:56.223 INFO:teuthology.orchestra.run.smithi115.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 166 kB/s | 125 kB 00:00 2024-09-10T19:08:56.283 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T19:08:56.307 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:56.332 INFO:teuthology.orchestra.run.smithi115.stdout:(12/34): python3-markupsafe-1.1.1-12.el9.x86_64 321 kB/s | 35 kB 00:00 2024-09-10T19:08:56.357 INFO:teuthology.orchestra.run.smithi115.stdout:(13/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-10T19:08:56.423 INFO:teuthology.orchestra.run.smithi115.stdout:(14/34): python3-pytz-2021.1-5.el9.noarch.rpm 768 kB/s | 51 kB 00:00 2024-09-10T19:08:56.449 INFO:teuthology.orchestra.run.smithi115.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 659 kB/s | 77 kB 00:00 2024-09-10T19:08:56.482 INFO:teuthology.orchestra.run.smithi115.stdout:(16/34): python3-urllib3-1.26.5-6.el9.noarch.rp 289 kB/s | 215 kB 00:00 2024-09-10T19:08:56.508 INFO:teuthology.orchestra.run.smithi115.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-10T19:08:56.517 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T19:08:56.540 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:56.541 INFO:teuthology.orchestra.run.smithi115.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-09-10T19:08:56.575 INFO:teuthology.orchestra.run.smithi115.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 446 kB/s | 11 kB 00:00 2024-09-10T19:08:56.600 INFO:teuthology.orchestra.run.smithi115.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-10T19:08:56.605 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T19:08:56.626 INFO:teuthology.orchestra.run.smithi115.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 366 kB/s | 18 kB 00:00 2024-09-10T19:08:56.651 INFO:teuthology.orchestra.run.smithi115.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-10T19:08:56.676 INFO:teuthology.orchestra.run.smithi115.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-10T19:08:56.701 INFO:teuthology.orchestra.run.smithi115.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-10T19:08:56.727 INFO:teuthology.orchestra.run.smithi115.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-10T19:08:56.753 INFO:teuthology.orchestra.run.smithi115.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T19:08:56.777 INFO:teuthology.orchestra.run.smithi115.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-10T19:08:56.803 INFO:teuthology.orchestra.run.smithi115.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-10T19:08:56.828 INFO:teuthology.orchestra.run.smithi115.stdout:(29/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T19:08:56.853 INFO:teuthology.orchestra.run.smithi115.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-10T19:08:56.878 INFO:teuthology.orchestra.run.smithi115.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-10T19:08:56.903 INFO:teuthology.orchestra.run.smithi115.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-10T19:08:56.929 INFO:teuthology.orchestra.run.smithi115.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T19:08:56.954 INFO:teuthology.orchestra.run.smithi115.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-10T19:08:56.961 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:08:56.961 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.0 MB/s | 27 MB 00:03 2024-09-10T19:08:57.091 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:08:57.127 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:08:57.128 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:08:57.498 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:08:57.498 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:08:58.346 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T19:08:58.347 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T19:08:58.348 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T19:08:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T19:08:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T19:08:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T19:08:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T19:08:58.349 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T19:08:58.350 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T19:08:58.751 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:08:58.824 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T19:08:58.866 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T19:08:58.908 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T19:08:58.999 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T19:08:59.094 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T19:08:59.135 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T19:08:59.170 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T19:08:59.200 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T19:08:59.234 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T19:08:59.268 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T19:08:59.308 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T19:08:59.343 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T19:08:59.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:59.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T19:08:59.402 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T19:08:59.403 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T19:08:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T19:08:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T19:08:59.404 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:08:59.404 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:08:59.426 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T19:08:59.466 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T19:08:59.541 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T19:08:59.686 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T19:08:59.774 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T19:08:59.776 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-base 2024-09-10T19:08:59.828 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T19:08:59.866 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T19:08:59.914 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T19:08:59.971 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T19:09:00.165 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T19:09:00.220 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T19:09:00.289 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T19:09:00.351 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:00.378 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T19:09:00.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T19:09:00.441 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:00.482 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:00.483 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:09:00.484 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:00.534 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T19:09:00.561 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install cephadm 2024-09-10T19:09:00.689 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T19:09:00.721 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T19:09:00.746 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T19:09:00.746 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:00.746 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T19:09:00.746 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T19:09:00.747 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T19:09:00.747 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:01.128 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:01.262 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:01.262 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 83 k 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 375 k 2024-09-10T19:09:01.263 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:01.451 INFO:teuthology.orchestra.run.smithi150.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 441 kB/s | 83 kB 00:00 2024-09-10T19:09:01.452 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:01.452 INFO:teuthology.orchestra.run.smithi150.stdout:Total 438 kB/s | 83 kB 00:00 2024-09-10T19:09:01.452 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:01.457 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:01.457 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:01.459 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:01.459 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:01.526 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:01.925 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:02.004 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:02.131 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T19:09:02.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:02.357 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:02.532 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T19:09:02.556 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T19:09:02.556 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:02.556 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T19:09:02.556 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T19:09:02.556 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T19:09:02.557 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:02.617 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:02.618 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:02.618 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:02.618 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:02.761 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T19:09:02.781 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:02.845 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T19:09:02.854 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T19:09:03.434 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:03.569 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:03.570 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 137 k 2024-09-10T19:09:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 379 k 2024-09-10T19:09:03.571 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:03.782 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 650 kB/s | 137 kB 00:00 2024-09-10T19:09:03.783 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:03.783 INFO:teuthology.orchestra.run.smithi150.stdout:Total 645 kB/s | 137 kB 00:00 2024-09-10T19:09:03.783 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:03.790 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:03.791 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:03.852 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:03.853 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:04.013 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:04.075 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:04.100 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:04.100 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T19:09:04.100 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T19:09:04.281 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T19:09:04.282 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T19:09:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T19:09:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T19:09:04.283 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T19:09:04.284 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T19:09:05.060 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:05.060 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:05.060 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:05.061 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.061 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:05.061 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.084 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T19:09:05.085 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T19:09:05.086 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:05.087 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:05.311 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr 2024-09-10T19:09:05.495 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2024-09-10T19:09:05.892 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:05.988 INFO:teuthology.orchestra.run.smithi150.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:06.029 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:06.030 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:09:06.030 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:06.069 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:06.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T19:09:06.160 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:06.202 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:06.203 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:06.203 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:06.264 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2024-09-10T19:09:06.669 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:06.802 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:06.802 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:06.802 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:06.802 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout:Installing weak dependencies: 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:06.803 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:06.804 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:06.804 INFO:teuthology.orchestra.run.smithi150.stdout:Install 7 Packages 2024-09-10T19:09:06.804 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:06.804 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 3.7 M 2024-09-10T19:09:06.805 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 80 M 2024-09-10T19:09:06.805 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:06.825 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:06.954 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T19:09:06.955 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 83 k 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 375 k 2024-09-10T19:09:06.956 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:07.111 INFO:teuthology.orchestra.run.smithi150.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 166 kB/s | 24 kB 00:00 2024-09-10T19:09:07.136 INFO:teuthology.orchestra.run.smithi150.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 85 kB/s | 15 kB 00:00 2024-09-10T19:09:07.140 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 450 kB/s | 83 kB 00:00 2024-09-10T19:09:07.140 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:07.141 INFO:teuthology.orchestra.run.smithi115.stdout:Total 446 kB/s | 83 kB 00:00 2024-09-10T19:09:07.141 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:07.146 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:07.146 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:07.148 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:07.149 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:07.178 INFO:teuthology.orchestra.run.smithi150.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 136 kB/s | 9.0 kB 00:00 2024-09-10T19:09:07.203 INFO:teuthology.orchestra.run.smithi150.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T19:09:07.223 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:07.228 INFO:teuthology.orchestra.run.smithi150.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-10T19:09:07.254 INFO:teuthology.orchestra.run.smithi150.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T19:09:07.379 INFO:teuthology.orchestra.run.smithi150.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.1 MB/s | 3.4 MB 00:00 2024-09-10T19:09:07.382 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:07.382 INFO:teuthology.orchestra.run.smithi150.stdout:Total 6.3 MB/s | 3.7 MB 00:00 2024-09-10T19:09:07.383 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:07.391 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:07.391 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:07.473 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:07.474 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:07.682 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:07.753 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:07.772 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:07.847 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T19:09:07.881 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T19:09:07.935 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T19:09:07.967 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T19:09:07.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T19:09:08.122 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:08.369 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:08.563 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T19:09:08.994 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T19:09:09.042 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T19:09:09.129 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:58 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:09.262 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:09.263 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T19:09:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:09.263 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:09.263 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 137 k 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 379 k 2024-09-10T19:09:09.264 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:09.464 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 688 kB/s | 137 kB 00:00 2024-09-10T19:09:09.465 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:09.465 INFO:teuthology.orchestra.run.smithi115.stdout:Total 681 kB/s | 137 kB 00:00 2024-09-10T19:09:09.465 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:09.473 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:09.473 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:09.535 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:09.535 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:09.701 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:09.776 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:09.799 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:09.799 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:09.799 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T19:09:09.799 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T19:09:10.135 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T19:09:10.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T19:09:10.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T19:09:10.409 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:10.409 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:10.632 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T19:09:10.710 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T19:09:10.710 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:10.710 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:10.710 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:10.711 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:10.711 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:10.951 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2024-09-10T19:09:11.194 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:11.327 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:11.328 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T19:09:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:11.328 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T19:09:11.328 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:11.328 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T19:09:11.329 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout:======================================================================================= 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout:Install 12 Packages 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:11.330 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 38 M 2024-09-10T19:09:11.331 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 212 M 2024-09-10T19:09:11.331 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:11.520 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:11.614 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:11.655 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:11.656 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:11.656 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:11.711 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T19:09:11.962 INFO:teuthology.orchestra.run.smithi150.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-10T19:09:12.054 INFO:teuthology.orchestra.run.smithi150.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 345 kB/s | 191 kB 00:00 2024-09-10T19:09:12.262 INFO:teuthology.orchestra.run.smithi150.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 109 kB/s | 33 kB 00:00 2024-09-10T19:09:12.287 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:01 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:12.321 INFO:teuthology.orchestra.run.smithi150.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 975 kB/s | 801 kB 00:00 2024-09-10T19:09:12.371 INFO:teuthology.orchestra.run.smithi150.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 167 kB/s | 18 kB 00:00 2024-09-10T19:09:12.421 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T19:09:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:12.423 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.7 M 2024-09-10T19:09:12.424 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 80 M 2024-09-10T19:09:12.424 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:12.430 INFO:teuthology.orchestra.run.smithi150.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 354 kB/s | 38 kB 00:00 2024-09-10T19:09:12.547 INFO:teuthology.orchestra.run.smithi150.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-10T19:09:12.610 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 159 kB/s | 24 kB 00:00 2024-09-10T19:09:12.635 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 82 kB/s | 15 kB 00:00 2024-09-10T19:09:12.678 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T19:09:12.819 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T19:09:12.819 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T19:09:12.820 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T19:09:12.822 INFO:teuthology.orchestra.run.smithi150.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.9 MB/s | 3.0 MB 00:00 2024-09-10T19:09:12.853 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.4 MB/s | 3.4 MB 00:00 2024-09-10T19:09:12.856 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:12.857 INFO:teuthology.orchestra.run.smithi115.stdout:Total 8.4 MB/s | 3.7 MB 00:00 2024-09-10T19:09:12.857 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:12.865 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:12.865 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:12.947 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:12.947 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:12.948 INFO:teuthology.orchestra.run.smithi150.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-10T19:09:13.251 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:13.312 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T19:09:13.347 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T19:09:13.402 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T19:09:13.433 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T19:09:13.448 INFO:teuthology.orchestra.run.smithi150.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.6 MB/s | 4.9 MB 00:01 2024-09-10T19:09:13.462 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T19:09:14.460 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T19:09:14.499 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T19:09:15.048 INFO:teuthology.orchestra.run.smithi150.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.0 MB/s | 5.1 MB 00:02 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T19:09:15.509 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:15.792 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T19:09:15.793 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T19:09:15.793 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T19:09:15.793 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T19:09:15.793 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:15.793 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:16.007 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T19:09:16.571 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:05 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:16.704 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T19:09:16.705 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================= 2024-09-10T19:09:16.706 INFO:teuthology.orchestra.run.smithi115.stdout:Install 12 Packages 2024-09-10T19:09:16.707 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:16.707 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 38 M 2024-09-10T19:09:16.707 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 212 M 2024-09-10T19:09:16.707 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:17.294 INFO:teuthology.orchestra.run.smithi115.stdout:(1/12): ceph-mgr-diskprediction-local-17.2.7-16 17 MB/s | 7.4 MB 00:00 2024-09-10T19:09:17.345 INFO:teuthology.orchestra.run.smithi115.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 388 kB/s | 191 kB 00:00 2024-09-10T19:09:17.553 INFO:teuthology.orchestra.run.smithi115.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-10T19:09:17.578 INFO:teuthology.orchestra.run.smithi115.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 115 kB/s | 33 kB 00:00 2024-09-10T19:09:17.820 INFO:teuthology.orchestra.run.smithi115.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 68 kB/s | 18 kB 00:00 2024-09-10T19:09:17.845 INFO:teuthology.orchestra.run.smithi115.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 144 kB/s | 38 kB 00:00 2024-09-10T19:09:18.346 INFO:teuthology.orchestra.run.smithi115.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 489 kB/s | 245 kB 00:00 2024-09-10T19:09:18.463 INFO:teuthology.orchestra.run.smithi115.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-10T19:09:18.730 INFO:teuthology.orchestra.run.smithi115.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 5.4 MB/s | 4.9 MB 00:00 2024-09-10T19:09:18.963 INFO:teuthology.orchestra.run.smithi115.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 501 kB/s | 250 kB 00:00 2024-09-10T19:09:19.338 INFO:teuthology.orchestra.run.smithi115.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.1 MB/s | 5.1 MB 00:00 2024-09-10T19:09:20.122 INFO:teuthology.orchestra.run.smithi115.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-10T19:09:20.123 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:20.123 INFO:teuthology.orchestra.run.smithi115.stdout:Total 11 MB/s | 38 MB 00:03 2024-09-10T19:09:20.482 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:20.508 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:20.508 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:21.021 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:21.022 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:21.798 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:21.863 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T19:09:21.919 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T19:09:22.017 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T19:09:22.071 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T19:09:22.256 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T19:09:22.295 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T19:09:22.389 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T19:09:23.029 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T19:09:23.148 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T19:09:24.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T19:09:25.401 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T19:09:25.428 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T19:09:28.008 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T19:09:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T19:09:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T19:09:28.009 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T19:09:28.450 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:28.451 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:28.835 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2024-09-10T19:09:29.413 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:18 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:29.548 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T19:09:29.549 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:29.550 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:29.551 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2024-09-10T19:09:29.551 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:29.551 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.2 M 2024-09-10T19:09:29.551 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 26 M 2024-09-10T19:09:29.551 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:30.453 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 171 kB/s | 26 kB 00:00 2024-09-10T19:09:30.478 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 271 kB/s | 47 kB 00:00 2024-09-10T19:09:30.511 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 92 kB/s | 19 kB 00:00 2024-09-10T19:09:30.603 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-10T19:09:30.629 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-10T19:09:30.654 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T19:09:30.679 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 639 kB/s | 32 kB 00:00 2024-09-10T19:09:30.705 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 290 kB/s | 14 kB 00:00 2024-09-10T19:09:30.738 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-10T19:09:30.764 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-09-10T19:09:30.789 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-10T19:09:30.839 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-10T19:09:30.846 INFO:teuthology.orchestra.run.smithi150.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 930 kB/s | 16 MB 00:17 2024-09-10T19:09:30.848 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:30.848 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.0 MB/s | 38 MB 00:19 2024-09-10T19:09:30.881 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-10T19:09:30.884 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:30.884 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-09-10T19:09:30.948 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:30.964 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:30.964 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:31.051 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:31.052 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:31.206 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:31.232 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:31.549 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:31.725 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T19:09:31.754 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:31.755 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:31.781 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T19:09:31.836 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T19:09:31.866 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T19:09:31.907 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T19:09:32.007 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T19:09:32.118 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T19:09:32.175 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T19:09:32.509 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:32.577 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T19:09:32.621 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T19:09:32.718 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T19:09:32.754 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T19:09:32.891 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T19:09:32.939 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T19:09:32.948 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T19:09:32.963 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T19:09:32.990 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T19:09:33.054 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T19:09:33.060 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T19:09:33.076 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T19:09:33.717 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T19:09:33.840 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T19:09:33.897 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T19:09:33.897 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T19:09:33.897 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T19:09:33.897 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T19:09:33.898 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T19:09:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T19:09:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:34.231 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:34.231 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:34.232 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:34.484 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T19:09:35.047 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:24 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:35.074 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T19:09:35.179 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:35.179 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:35.179 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:35.179 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2024-09-10T19:09:35.180 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:35.181 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 7.0 M 2024-09-10T19:09:35.181 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 33 M 2024-09-10T19:09:35.181 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:35.675 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 610 kB/s | 124 kB 00:00 2024-09-10T19:09:35.725 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T19:09:35.750 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T19:09:35.776 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T19:09:36.001 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 469 kB/s | 248 kB 00:00 2024-09-10T19:09:36.108 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T19:09:36.144 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T19:09:37.068 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-09-10T19:09:37.070 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:37.070 INFO:teuthology.orchestra.run.smithi115.stdout:Total 3.7 MB/s | 7.0 MB 00:01 2024-09-10T19:09:37.143 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:37.155 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:37.155 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:37.255 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:37.255 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:37.554 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:37.714 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T19:09:37.769 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T19:09:38.279 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T19:09:38.372 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T19:09:38.428 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T19:09:38.457 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T19:09:38.699 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T19:09:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T19:09:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T19:09:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T19:09:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T19:09:38.700 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T19:09:38.701 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T19:09:39.071 INFO:teuthology.orchestra.run.smithi150.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:39.072 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:39.272 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T19:09:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T19:09:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T19:09:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T19:09:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T19:09:39.273 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T19:09:39.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-rook 2024-09-10T19:09:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T19:09:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:39.561 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:39.561 INFO:teuthology.orchestra.run.smithi115.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:39.562 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:39.814 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2024-09-10T19:09:40.041 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:40.176 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:40.177 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:40.177 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:40.177 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:40.177 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T19:09:40.178 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout:Install 13 Packages 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.2 M 2024-09-10T19:09:40.179 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 26 M 2024-09-10T19:09:40.180 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:40.376 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:40.508 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:40.508 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:40.508 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:40.508 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:40.512 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:40.512 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T19:09:40.512 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2024-09-10T19:09:40.512 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T19:09:40.513 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:40.513 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:40.513 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:40.513 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2024-09-10T19:09:40.513 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:40.514 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 893 k 2024-09-10T19:09:40.514 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 2.6 M 2024-09-10T19:09:40.514 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:40.611 INFO:teuthology.orchestra.run.smithi150.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 163 kB/s | 19 kB 00:00 2024-09-10T19:09:40.636 INFO:teuthology.orchestra.run.smithi150.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 180 kB/s | 26 kB 00:00 2024-09-10T19:09:40.662 INFO:teuthology.orchestra.run.smithi150.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 283 kB/s | 47 kB 00:00 2024-09-10T19:09:40.711 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 554 kB/s | 80 kB 00:00 2024-09-10T19:09:40.770 INFO:teuthology.orchestra.run.smithi150.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-10T19:09:40.804 INFO:teuthology.orchestra.run.smithi150.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 948 kB/s | 159 kB 00:00 2024-09-10T19:09:40.829 INFO:teuthology.orchestra.run.smithi150.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 913 kB/s | 54 kB 00:00 2024-09-10T19:09:40.861 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-10T19:09:40.861 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:40.862 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.5 MB/s | 893 kB 00:00 2024-09-10T19:09:40.871 INFO:teuthology.orchestra.run.smithi150.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-10T19:09:40.873 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:40.895 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:40.895 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:40.905 INFO:teuthology.orchestra.run.smithi150.stdout:(8/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-10T19:09:40.930 INFO:teuthology.orchestra.run.smithi150.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 141 kB/s | 14 kB 00:00 2024-09-10T19:09:40.964 INFO:teuthology.orchestra.run.smithi150.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.2 MB/s | 59 kB 00:00 2024-09-10T19:09:40.974 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:40.974 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:40.989 INFO:teuthology.orchestra.run.smithi150.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-10T19:09:41.023 INFO:teuthology.orchestra.run.smithi150.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-10T19:09:41.081 INFO:teuthology.orchestra.run.smithi150.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-10T19:09:41.084 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:41.084 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-09-10T19:09:41.148 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:41.154 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:41.163 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:41.163 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:41.252 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:41.252 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:41.290 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T19:09:41.351 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T19:09:41.754 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:41.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T19:09:41.974 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T19:09:42.037 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T19:09:42.075 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T19:09:42.101 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T19:09:42.102 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T19:09:42.124 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T19:09:42.245 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T19:09:42.370 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:42.397 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:42.426 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T19:09:42.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-volume 2024-09-10T19:09:43.138 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T19:09:43.190 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T19:09:43.224 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T19:09:43.232 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:43.270 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T19:09:43.294 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T19:09:43.368 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:43.393 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:43.393 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2024-09-10T19:09:43.393 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 270 k 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 1.3 M 2024-09-10T19:09:43.394 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:43.590 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T19:09:43.591 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:43.591 INFO:teuthology.orchestra.run.smithi115.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T19:09:43.591 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:43.599 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:43.600 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:43.627 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:43.627 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:43.838 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:43.900 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:43.916 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:43.916 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:43.916 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T19:09:43.916 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T19:09:44.114 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T19:09:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T19:09:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T19:09:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T19:09:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T19:09:44.115 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T19:09:44.462 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:44.463 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:44.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T19:09:44.938 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:44.939 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:44.939 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:44.939 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:45.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2024-09-10T19:09:45.288 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:45.420 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:45.420 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:45.420 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:45.420 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout:Install 6 Packages 2024-09-10T19:09:45.421 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:45.422 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 7.0 M 2024-09-10T19:09:45.422 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 33 M 2024-09-10T19:09:45.422 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:45.813 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:45.875 INFO:teuthology.orchestra.run.smithi150.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 566 kB/s | 124 kB 00:00 2024-09-10T19:09:45.900 INFO:teuthology.orchestra.run.smithi150.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-10T19:09:45.946 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:45.947 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:45.948 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 126 k 2024-09-10T19:09:45.948 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 448 k 2024-09-10T19:09:45.948 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:45.950 INFO:teuthology.orchestra.run.smithi150.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T19:09:45.975 INFO:teuthology.orchestra.run.smithi150.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T19:09:46.000 INFO:teuthology.orchestra.run.smithi150.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.3 MB/s | 548 kB 00:00 2024-09-10T19:09:46.129 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 697 kB/s | 126 kB 00:00 2024-09-10T19:09:46.130 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:46.130 INFO:teuthology.orchestra.run.smithi115.stdout:Total 691 kB/s | 126 kB 00:00 2024-09-10T19:09:46.130 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:46.135 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:46.136 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:46.199 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:46.199 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:46.365 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:46.451 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:46.886 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:47.161 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:47.396 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2024-09-10T19:09:47.968 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:48.059 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:48.104 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:48.105 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:48.105 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:48.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2024-09-10T19:09:48.184 INFO:teuthology.orchestra.run.smithi150.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-09-10T19:09:48.186 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:48.186 INFO:teuthology.orchestra.run.smithi150.stdout:Total 2.5 MB/s | 7.0 MB 00:02 2024-09-10T19:09:48.258 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:48.270 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:48.270 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:48.370 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:48.370 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:48.682 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:48.738 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:48.861 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T19:09:48.870 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:48.871 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T19:09:48.871 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:48.871 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:48.871 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 25 k 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 110 k 2024-09-10T19:09:48.872 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:48.911 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T19:09:49.032 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 156 kB/s | 25 kB 00:00 2024-09-10T19:09:49.033 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:49.033 INFO:teuthology.orchestra.run.smithi115.stdout:Total 154 kB/s | 25 kB 00:00 2024-09-10T19:09:49.033 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:49.036 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:49.036 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:49.037 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:49.038 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:49.123 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:49.246 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:49.434 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T19:09:49.522 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T19:09:49.577 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T19:09:49.607 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T19:09:49.608 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:49.841 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:49.842 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:49.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:49.842 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:49.995 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T19:09:50.395 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T19:09:50.558 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:50.647 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:50.669 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T19:09:50.669 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:50.669 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:50.670 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:50.690 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:50.691 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:50.691 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:50.765 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2024-09-10T19:09:50.923 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-fuse 2024-09-10T19:09:51.331 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:51.421 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:51.463 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:51.464 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:51.465 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:51.487 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:51.538 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2024-09-10T19:09:51.619 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:51.619 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:51.619 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:51.619 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout:Install 2 Packages 2024-09-10T19:09:51.620 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:51.621 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 893 k 2024-09-10T19:09:51.621 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 2.6 M 2024-09-10T19:09:51.621 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:51.752 INFO:teuthology.orchestra.run.smithi150.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 720 kB/s | 80 kB 00:00 2024-09-10T19:09:51.911 INFO:teuthology.orchestra.run.smithi150.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-10T19:09:51.911 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:51.912 INFO:teuthology.orchestra.run.smithi150.stdout:Total 3.0 MB/s | 893 kB 00:00 2024-09-10T19:09:51.923 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:51.945 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:51.945 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:52.024 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:52.025 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:52.095 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:52.184 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:52.212 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:52.226 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:52.227 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:52.227 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:52.295 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2024-09-10T19:09:52.340 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T19:09:52.404 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T19:09:52.854 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:52.944 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:52.987 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:52.988 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:52.988 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:53.056 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2024-09-10T19:09:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T19:09:53.070 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T19:09:53.333 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T19:09:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:53.334 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T19:09:53.334 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:53.334 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:53.623 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:53.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-volume 2024-09-10T19:09:53.712 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:53.755 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:53.756 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:53.756 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:53.836 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2024-09-10T19:09:54.204 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:54.342 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:09:54.343 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:54.344 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 270 k 2024-09-10T19:09:54.344 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 1.3 M 2024-09-10T19:09:54.344 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:54.402 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:54.492 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:54.535 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:54.536 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2024-09-10T19:09:54.536 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:54.574 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-10T19:09:54.575 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:54.575 INFO:teuthology.orchestra.run.smithi150.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T19:09:54.575 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:54.583 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:54.584 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:54.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2024-09-10T19:09:54.611 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:54.612 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:54.827 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:54.893 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:54.909 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:54.909 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:54.909 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T19:09:54.910 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:55.166 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:55.298 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:55.298 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:55.298 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T19:09:55.298 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 82 k 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 222 k 2024-09-10T19:09:55.299 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:55.496 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 421 kB/s | 82 kB 00:00 2024-09-10T19:09:55.496 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:55.497 INFO:teuthology.orchestra.run.smithi115.stdout:Total 418 kB/s | 82 kB 00:00 2024-09-10T19:09:55.497 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:55.501 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:55.501 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:55.564 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:55.564 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:55.711 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:55.806 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:55.891 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:56.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados-devel 2024-09-10T19:09:56.197 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:56.476 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:56.701 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2024-09-10T19:09:56.753 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:56.887 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:56.888 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:09:56.889 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:56.889 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 126 k 2024-09-10T19:09:56.889 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 448 k 2024-09-10T19:09:56.889 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:09:57.096 INFO:teuthology.orchestra.run.smithi150.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 609 kB/s | 126 kB 00:00 2024-09-10T19:09:57.097 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:57.097 INFO:teuthology.orchestra.run.smithi150.stdout:Total 605 kB/s | 126 kB 00:00 2024-09-10T19:09:57.097 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:09:57.102 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:09:57.102 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:09:57.166 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:09:57.166 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:09:57.275 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:57.325 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:09:57.411 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T19:09:57.412 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:57.413 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:09:57.413 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:09:57.413 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:09:57.413 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:57.414 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 2.9 M 2024-09-10T19:09:57.414 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 12 M 2024-09-10T19:09:57.415 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:09:57.426 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:57.738 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-10T19:09:57.739 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:09:57.739 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-10T19:09:57.739 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:09:57.748 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:09:57.748 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:09:57.829 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:09:57.829 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:09:57.866 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:58.119 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:58.119 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:58.119 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:09:58.119 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:58.119 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:58.120 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:58.238 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:09:58.305 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:58.330 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:58.330 INFO:teuthology.orchestra.run.smithi115.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:09:58.330 INFO:teuthology.orchestra.run.smithi115.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T19:09:58.331 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T19:09:58.331 INFO:teuthology.orchestra.run.smithi115.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T19:09:58.331 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:58.387 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs2 2024-09-10T19:09:58.969 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:47 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:59.062 INFO:teuthology.orchestra.run.smithi150.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:09:59.107 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:59.108 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:09:59.108 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:09:59.210 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install libcephfs-devel 2024-09-10T19:09:59.401 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:09:59.401 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:59.401 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:09:59.401 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:09:59.401 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:09:59.402 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:09:59.619 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2024-09-10T19:09:59.781 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:48 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:09:59.915 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:09:59.916 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T19:09:59.916 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:59.916 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:09:59.916 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 25 k 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 110 k 2024-09-10T19:09:59.917 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:10:00.075 INFO:teuthology.orchestra.run.smithi150.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 158 kB/s | 25 kB 00:00 2024-09-10T19:10:00.076 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:00.076 INFO:teuthology.orchestra.run.smithi150.stdout:Total 157 kB/s | 25 kB 00:00 2024-09-10T19:10:00.076 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:10:00.079 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:10:00.079 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:10:00.081 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:10:00.081 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:10:00.169 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:10:00.193 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:00.273 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:00.326 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2024-09-10T19:10:00.327 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:10:00.327 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2024-09-10T19:10:00.327 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:10:00.327 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 165 k 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 477 k 2024-09-10T19:10:00.328 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2024-09-10T19:10:00.541 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 775 kB/s | 165 kB 00:00 2024-09-10T19:10:00.542 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:00.542 INFO:teuthology.orchestra.run.smithi115.stdout:Total 769 kB/s | 165 kB 00:00 2024-09-10T19:10:00.542 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2024-09-10T19:10:00.550 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2024-09-10T19:10:00.550 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2024-09-10T19:10:00.613 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2024-09-10T19:10:00.613 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2024-09-10T19:10:00.644 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:00.762 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2024-09-10T19:10:00.854 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:00.866 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:00.866 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:00.866 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:10:00.866 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:10:00.866 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:00.867 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:01.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librados2 2024-09-10T19:10:01.313 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:01.574 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:01.574 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:10:01.574 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2024-09-10T19:10:01.574 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:10:01.575 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:10:01.575 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2024-09-10T19:10:01.649 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:50 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:01.739 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:01.783 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:01.783 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:01.783 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:01.812 DEBUG:teuthology.parallel:result is None 2024-09-10T19:10:01.859 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install librbd1 2024-09-10T19:10:02.428 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:51 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:02.516 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:02.560 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:02.561 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:02.561 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:02.635 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rados 2024-09-10T19:10:03.205 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:52 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:03.296 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:03.340 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:03.341 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:03.341 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:03.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rgw 2024-09-10T19:10:03.992 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:52 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:04.083 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:04.125 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:04.126 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:04.126 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:04.201 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-cephfs 2024-09-10T19:10:04.770 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:53 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:04.860 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:04.903 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:04.903 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:04.904 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:04.984 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install python3-rbd 2024-09-10T19:10:05.548 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:54 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:05.639 INFO:teuthology.orchestra.run.smithi150.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T19:10:05.681 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:05.682 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2024-09-10T19:10:05.682 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:05.760 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-fuse 2024-09-10T19:10:06.339 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:55 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:06.480 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:06.480 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:06.480 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 82 k 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 222 k 2024-09-10T19:10:06.481 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:10:06.687 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 397 kB/s | 82 kB 00:00 2024-09-10T19:10:06.688 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:06.689 INFO:teuthology.orchestra.run.smithi150.stdout:Total 393 kB/s | 82 kB 00:00 2024-09-10T19:10:06.689 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:10:06.696 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:10:06.696 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:10:06.762 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:10:06.763 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:10:06.916 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:10:07.019 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:07.467 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:07.716 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:07.717 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:07.955 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-mirror 2024-09-10T19:10:08.537 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:57 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:08.675 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:08.676 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:08.676 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repo Size 2024-09-10T19:10:08.676 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:08.676 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:10:08.676 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 2.9 M 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 12 M 2024-09-10T19:10:08.677 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:10:08.999 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 9.1 MB/s | 2.9 MB 00:00 2024-09-10T19:10:09.000 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:09.000 INFO:teuthology.orchestra.run.smithi150.stdout:Total 9.1 MB/s | 2.9 MB 00:00 2024-09-10T19:10:09.000 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:10:09.009 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:10:09.009 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:10:09.089 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:10:09.089 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:10:09.506 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:10:09.566 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:09.592 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:09.593 INFO:teuthology.orchestra.run.smithi150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T19:10:09.593 INFO:teuthology.orchestra.run.smithi150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T19:10:09.593 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T19:10:09.593 INFO:teuthology.orchestra.run.smithi150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T19:10:09.593 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:10.752 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:10.753 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:10.995 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install rbd-nbd 2024-09-10T19:10:11.573 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:02:00 ago on Tue 10 Sep 2024 07:08:11 PM UTC. 2024-09-10T19:10:11.709 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout: Package Architecture Version Repository Size 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:================================================================================ 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout:Install 1 Package 2024-09-10T19:10:11.710 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:11.711 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 165 k 2024-09-10T19:10:11.711 INFO:teuthology.orchestra.run.smithi150.stdout:Installed size: 477 k 2024-09-10T19:10:11.711 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2024-09-10T19:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 758 kB/s | 165 kB 00:00 2024-09-10T19:10:11.929 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout:Total 752 kB/s | 165 kB 00:00 2024-09-10T19:10:11.930 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2024-09-10T19:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2024-09-10T19:10:11.937 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2024-09-10T19:10:12.000 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2024-09-10T19:10:12.000 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2024-09-10T19:10:12.161 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2024-09-10T19:10:12.259 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:12.728 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:12.989 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T19:10:12.990 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:12.990 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2024-09-10T19:10:12.990 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T19:10:12.990 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:10:12.990 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2024-09-10T19:10:13.282 DEBUG:teuthology.parallel:result is None 2024-09-10T19:10:13.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:10:13.426 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T19:10:13.452 INFO:teuthology.orchestra.run.smithi115.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T19:10:13.452 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T19:10:13.452 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T19:10:13.455 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:10:13.597 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T19:10:13.620 INFO:teuthology.orchestra.run.smithi150.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T19:10:13.620 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T19:10:13.620 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T19:10:13.622 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T19:10:13.623 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:10:13.623 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T19:10:13.654 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:10:13.654 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T19:10:13.686 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T19:10:13.686 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:10:13.686 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T19:10:13.722 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T19:10:13.791 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:10:13.791 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T19:10:13.817 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T19:10:13.882 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T19:10:13.882 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:10:13.882 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T19:10:13.907 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T19:10:13.973 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:10:13.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T19:10:13.999 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T19:10:14.084 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T19:10:14.084 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:10:14.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T19:10:14.110 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T19:10:14.178 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:10:14.178 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T19:10:14.203 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T19:10:14.270 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T19:10:14.373 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'cephadm-package'} 2024-09-10T19:10:14.373 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:10:14.373 INFO:tasks.cephadm:Cluster fsid is 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:14.374 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T19:10:14.374 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.115:6789]', 'mon.c': '[v1:172.21.15.115:6790]', 'mon.b': '[v1:172.21.15.150:6789]'} 2024-09-10T19:10:14.374 INFO:tasks.cephadm:First mon is mon.a on smithi115 2024-09-10T19:10:14.374 INFO:tasks.cephadm:First mgr is y 2024-09-10T19:10:14.374 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T19:10:14.374 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2024-09-10T19:10:14.401 DEBUG:teuthology.orchestra.run.smithi150:> sudo hostname $(hostname -s) 2024-09-10T19:10:14.441 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T19:10:14.441 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T19:10:14.443 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T19:10:15.948 INFO:teuthology.orchestra.run.smithi115.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T19:10:16.118 INFO:teuthology.orchestra.run.smithi150.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T19:10:42.714 INFO:teuthology.orchestra.run.smithi115.stdout:{ 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "repo_digests": [ 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout: ] 2024-09-10T19:10:42.715 INFO:teuthology.orchestra.run.smithi115.stdout:} 2024-09-10T19:10:45.506 INFO:teuthology.orchestra.run.smithi150.stdout:{ 2024-09-10T19:10:45.506 INFO:teuthology.orchestra.run.smithi150.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T19:10:45.506 INFO:teuthology.orchestra.run.smithi150.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T19:10:45.506 INFO:teuthology.orchestra.run.smithi150.stdout: "repo_digests": [ 2024-09-10T19:10:45.507 INFO:teuthology.orchestra.run.smithi150.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T19:10:45.507 INFO:teuthology.orchestra.run.smithi150.stdout: ] 2024-09-10T19:10:45.507 INFO:teuthology.orchestra.run.smithi150.stdout:} 2024-09-10T19:10:45.529 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2024-09-10T19:10:45.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph 2024-09-10T19:10:45.586 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2024-09-10T19:10:45.622 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /etc/ceph 2024-09-10T19:10:45.651 INFO:tasks.cephadm:Writing seed config... 2024-09-10T19:10:45.652 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T19:10:45.652 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T19:10:45.652 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-10T19:10:45.652 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T19:10:45.653 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T19:10:45.654 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:10:45.654 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T19:10:45.677 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 4f14fa52-6fa8-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T19:10:45.678 DEBUG:teuthology.orchestra.run.smithi115:mon.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T19:10:45.721 DEBUG:teuthology.orchestra.run.smithi115:mgr.y> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T19:10:45.764 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T19:10:45.764 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.115:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:10:45.962 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2024-09-10T19:10:45.962 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', '4f14fa52-6fa8-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.115:6789]', '--skip-admin-label'] 2024-09-10T19:10:45.986 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T19:10:45.987 INFO:teuthology.orchestra.run.smithi115.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T19:10:45.987 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying podman|docker is present... 2024-09-10T19:10:46.009 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T19:10:46.009 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying lvm2 is present... 2024-09-10T19:10:46.010 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying time synchronization is in place... 2024-09-10T19:10:46.017 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T19:10:46.017 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T19:10:46.024 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T19:10:46.024 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:10:46.030 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-09-10T19:10:46.037 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-09-10T19:10:46.037 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-09-10T19:10:46.037 INFO:teuthology.orchestra.run.smithi115.stdout:Repeating the final host check... 2024-09-10T19:10:46.059 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 5.2.2 2024-09-10T19:10:46.059 INFO:teuthology.orchestra.run.smithi115.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T19:10:46.059 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl is present 2024-09-10T19:10:46.059 INFO:teuthology.orchestra.run.smithi115.stdout:lvcreate is present 2024-09-10T19:10:46.066 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T19:10:46.066 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T19:10:46.072 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T19:10:46.072 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:10:46.079 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout enabled 2024-09-10T19:10:46.085 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout active 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Unit chronyd.service is enabled and running 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Host looks OK 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Cluster fsid: 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Acquiring lock 139701230314256 on /run/cephadm/4f14fa52-6fa8-11ef-bcea-c7b262605968.lock 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Lock 139701230314256 acquired on /run/cephadm/4f14fa52-6fa8-11ef-bcea-c7b262605968.lock 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying IP 172.21.15.115 port 6789 ... 2024-09-10T19:10:46.086 INFO:teuthology.orchestra.run.smithi115.stdout:Base mon IP(s) is [172.21.15.115:6789], mon addrv is [v1:172.21.15.115:6789] 2024-09-10T19:10:46.090 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.115 metric 100 2024-09-10T19:10:46.090 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.115 metric 100 2024-09-10T19:10:46.093 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T19:10:46.093 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T19:10:46.093 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link noprefixroute 2024-09-10T19:10:46.096 INFO:teuthology.orchestra.run.smithi115.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T19:10:46.097 INFO:teuthology.orchestra.run.smithi115.stdout:Mon IP `172.21.15.115` is in CIDR network `172.21.0.0/20` 2024-09-10T19:10:46.097 INFO:teuthology.orchestra.run.smithi115.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-10T19:10:46.097 INFO:teuthology.orchestra.run.smithi115.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T19:10:46.097 INFO:teuthology.orchestra.run.smithi115.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T19:10:46.498 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T19:10:46.498 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T19:10:46.498 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T19:10:46.499 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T19:10:46.499 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T19:10:46.499 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T19:10:46.499 INFO:teuthology.orchestra.run.smithi115.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T19:10:47.686 INFO:teuthology.orchestra.run.smithi115.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T19:10:47.686 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T19:10:47.686 INFO:teuthology.orchestra.run.smithi115.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T19:10:48.633 INFO:teuthology.orchestra.run.smithi115.stdout:stat: stdout 167 167 2024-09-10T19:10:48.633 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial keys... 2024-09-10T19:10:49.680 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQC4meBmRBGtORAAdII6njw5vCxXDPdkzTLUPg== 2024-09-10T19:10:50.569 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQC5meBmmlI2OBAA7kCQ045CqYLAUU7kCuzI+Q== 2024-09-10T19:10:51.491 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph-authtool: stdout AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== 2024-09-10T19:10:51.491 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial monmap... 2024-09-10T19:10:52.481 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:monmaptool for a [v1:172.21.15.115:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:setting min_mon_release = octopus 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: set fsid to 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:10:52.482 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mon... 2024-09-10T19:10:53.379 INFO:teuthology.orchestra.run.smithi115.stdout:create mon.a on 2024-09-10T19:10:53.579 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T19:10:53.723 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T19:10:53.866 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target. 2024-09-10T19:10:53.866 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target. 2024-09-10T19:10:54.036 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a 2024-09-10T19:10:54.036 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T19:10:54.198 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target.wants/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@.service. 2024-09-10T19:10:54.668 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T19:10:54.668 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon to start... 2024-09-10T19:10:54.675 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mon... 2024-09-10T19:10:54.789 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:54 smithi115 systemd[1]: Started Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:10:55.630 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout id: 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout services: 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.49211s) 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout data: 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T19:10:55.633 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T19:10:55.634 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T19:10:55.634 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:55.634 INFO:teuthology.orchestra.run.smithi115.stdout:mon is available 2024-09-10T19:10:55.634 INFO:teuthology.orchestra.run.smithi115.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T19:10:55.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T19:10:55.728 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: monmap e1: 1 mons at {a=v1:172.21.15.115:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:10:55.729 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: fsmap 2024-09-10T19:10:55.729 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T19:10:55.729 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: mgrmap e1: no daemons active 2024-09-10T19:10:55.729 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:55 smithi115 ceph-mon[28598]: from='client.? v1:172.21.15.115:0/1855340678' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T19:10:56.654 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:56.654 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T19:10:56.654 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:56.654 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.115:6789] 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T19:10:56.655 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T19:10:56.656 INFO:teuthology.orchestra.run.smithi115.stdout:Generating new minimal ceph.conf... 2024-09-10T19:10:56.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:56 smithi115 ceph-mon[28598]: from='client.? v1:172.21.15.115:0/2162856982' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T19:10:56.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:56 smithi115 ceph-mon[28598]: from='client.? v1:172.21.15.115:0/2162856982' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T19:10:57.659 INFO:teuthology.orchestra.run.smithi115.stdout:Restarting the monitor... 2024-09-10T19:10:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:57 smithi115 ceph-mon[28598]: from='client.? v1:172.21.15.115:0/241769466' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:10:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:57 smithi115 systemd[1]: Stopping Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:10:58.318 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:57 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a[28594]: 2024-09-10T19:10:57.941+0000 7fa94fc31640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:10:58.318 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:57 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a[28594]: 2024-09-10T19:10:57.941+0000 7fa94fc31640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T19:10:58.318 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 podman[28842]: 2024-09-10 19:10:58.064976937 +0000 UTC m=+0.246118549 container died 11baa05fceea19278860f59b5b009441c417b626df02095546efa92dc07f1875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T19:10:58.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 podman[28842]: 2024-09-10 19:10:58.399800905 +0000 UTC m=+0.580942546 container cleanup 11baa05fceea19278860f59b5b009441c417b626df02095546efa92dc07f1875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:10:58.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 bash[28842]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a 2024-09-10T19:10:58.571 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 podman[28854]: 2024-09-10 19:10:58.567227782 +0000 UTC m=+0.497342352 container remove 11baa05fceea19278860f59b5b009441c417b626df02095546efa92dc07f1875 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T19:10:58.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T19:10:58.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 systemd[1]: Stopped Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:10:58.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:58 smithi115 systemd[1]: Starting Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:10:59.296 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 podman[28949]: 2024-09-10 19:10:59.001341469 +0000 UTC m=+0.093755976 container create 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T19:10:59.296 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 podman[28949]: 2024-09-10 19:10:58.923424094 +0000 UTC m=+0.015838602 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 podman[28949]: 2024-09-10 19:10:59.220048046 +0000 UTC m=+0.312462554 container init 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 podman[28949]: 2024-09-10 19:10:59.223121594 +0000 UTC m=+0.315536104 container start 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: pidfile_write: ignore empty --pid-file 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: load: jerasure load: lrc 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: RocksDB version: 6.15.5 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Compile date Sep 9 2024 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: DB SUMMARY 2024-09-10T19:10:59.297 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: DB Session ID: K7HH01HIH1B7OR2G3311 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: CURRENT file: CURRENT 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82006 ; 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.error_if_exists: 0 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.create_if_missing: 0 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.paranoid_checks: 1 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.env: 0x558720570440 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.fs: Posix File System 2024-09-10T19:10:59.298 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.info_log: 0x558721fd4e60 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.statistics: (nil) 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.use_fsync: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_log_file_size: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_fallocate: 1 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.use_direct_reads: 0 2024-09-10T19:10:59.299 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.db_log_dir: 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T19:10:59.300 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.write_buffer_manager: 0x558722092780 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T19:10:59.301 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T19:10:59.302 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T19:10:59.302 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T19:10:59.302 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.unordered_write: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.row_cache: None 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.wal_filter: None 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.preserve_deletes: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.two_write_queues: 0 2024-09-10T19:10:59.303 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.atomic_flush: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.log_readahead_size: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T19:10:59.304 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_background_jobs: 2 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_background_compactions: -1 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_subcompactions: 1 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_open_files: -1 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T19:10:59.305 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_background_flushes: -1 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Compression algorithms supported: 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kZSTD supported: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kXpressCompression supported: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kBZip2Compression supported: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kLZ4Compression supported: 1 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kZlibCompression supported: 1 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T19:10:59.306 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: kSnappyCompression supported: 1 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.merge_operator: 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_filter: None 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558721f7ba38) 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-09-10T19:10:59.307 INFO:journalctl@ceph.mon.a.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_type: 0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_index_type: 0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_shortening: 1 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: hash_index_allow_collision: 1 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: checksum: 1 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: no_block_cache: 0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache: 0x558721fb11f0 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-09-10T19:10:59.308 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_options: 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: capacity : 536870912 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: num_shard_bits : 4 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: strict_capacity_limit : 0 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_cache_compressed: (nil) 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: persistent_cache: (nil) 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size: 4096 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_size_deviation: 10 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_restart_interval: 16 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: index_block_restart_interval: 1 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: metadata_block_size: 4096 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: partition_filters: 0 2024-09-10T19:10:59.309 INFO:journalctl@ceph.mon.a.smithi115.stdout: use_delta_encoding: 1 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: whole_key_filtering: 1 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: verify_compression: 0 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: format_version: 4 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_index_compression: 1 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout: block_align: 0 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression: NoCompression 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T19:10:59.310 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.num_levels: 7 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T19:10:59.311 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T19:10:59.312 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T19:10:59.313 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T19:10:59.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.inplace_update_support: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.bloom_locality: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.max_successive_merges: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.ttl: 2592000 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T19:10:59.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.enable_blob_files: false 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.min_blob_size: 0 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995459247412, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995459248138, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 78605, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 76867, "index_size": 200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9969, "raw_average_key_size": 47, "raw_value_size": 71189, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725995459, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "88cfb3fd-f7ea-495c-a9e7-1af2a892368c", "db_session_id": "K7HH01HIH1B7OR2G3311"}} 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T19:10:59.316 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995459248467, "job": 1, "event": "recovery_finished"} 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x558721fe4700 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: DB pointer 0x558721fb4000 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** DB Stats ** 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T19:10:59.317 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: L0 2/0 78.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: Sum 2/0 78.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.318 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative compaction: 0.00 GB write, 25.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval compaction: 0.00 GB write, 25.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:10:59.319 INFO:journalctl@ceph.mon.a.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: L0 2/0 78.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: Sum 2/0 78.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.5 0.00 0.00 1 0.000 0 0 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T19:10:59.320 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: Cumulative compaction: 0.00 GB write, 24.83 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: starting mon.a rank 0 at public addrs v1:172.21.15.115:6789/0 at bind addrs v1:172.21.15.115:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???) e1 preinit fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).mds e1 new map 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).mds e1 print_map 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: e1 2024-09-10T19:10:59.321 INFO:journalctl@ceph.mon.a.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout: legacy client fscid: -1 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout: 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout: No filesystems configured 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 bash[28949]: 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 2024-09-10T19:10:59.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 systemd[1]: Started Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:10:59.322 INFO:teuthology.orchestra.run.smithi115.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T19:10:59.557 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T19:10:59.557 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: monmap e1: 1 mons at {a=v1:172.21.15.115:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:10:59.557 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: fsmap 2024-09-10T19:10:59.557 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T19:10:59.557 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:10:59 smithi115 ceph-mon[28968]: mgrmap e1: no daemons active 2024-09-10T19:11:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T19:11:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:11:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:Creating mgr... 2024-09-10T19:11:00.364 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T19:11:00.365 INFO:teuthology.orchestra.run.smithi115.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T19:11:00.544 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y 2024-09-10T19:11:00.544 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Failed to reset failed state of unit ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T19:11:00.688 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968.target.wants/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@.service. 2024-09-10T19:11:00.999 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:00 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4146158567' entity='client.admin' 2024-09-10T19:11:01.211 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T19:11:01.211 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T19:11:01.218 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T19:11:01.219 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:11:01.219 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T19:11:01.219 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T19:11:01.227 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T19:11:01.227 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr to start... 2024-09-10T19:11:01.235 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr... 2024-09-10T19:11:01.272 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.270+0000 7fecec165ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:01.550 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.334+0000 7fecec165ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:11:01.550 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.462+0000 7fecec165ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:01.872 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4128612404' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T19:11:01.873 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.601+0000 7fecec165ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:11:01.873 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.870+0000 7fecec165ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:11:02.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:01 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:01.932+0000 7fecec165ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:11:02.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:02 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:02.095+0000 7fecec165ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "4f14fa52-6fa8-11ef-bcea-c7b262605968", 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.253 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.254 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T19:11:02.255 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T19:11:02.256 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T19:10:54.634387+0000", 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:02.257 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:02.258 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T19:11:02.258 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:02.258 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (1/15)... 2024-09-10T19:11:03.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:02 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:02.930+0000 7fecec165ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:11:03.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:02 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:02.994+0000 7fecec165ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:11:03.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:03.707+0000 7fecec165ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:11:03.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:03.815+0000 7fecec165ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:11:04.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:03.934+0000 7fecec165ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:04.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:03.992+0000 7fecec165ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:11:04.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.116+0000 7fecec165ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:11:04.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.181+0000 7fecec165ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:11:04.239 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.237+0000 7fecec165ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:11:04.501 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.385+0000 7fecec165ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:11:04.501 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.449+0000 7fecec165ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:11:04.754 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.516+0000 7fecec165ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:11:04.754 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.597+0000 7fecec165ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:04.754 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.657+0000 7fecec165ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:11:04.754 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:04.717+0000 7fecec165ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:11:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:04 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/695704207' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "4f14fa52-6fa8-11ef-bcea-c7b262605968", 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T19:11:05.300 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.301 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T19:11:05.302 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T19:11:05.303 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T19:10:54.634387+0000", 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:05.304 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:05.305 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T19:11:05.305 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:05.305 INFO:teuthology.orchestra.run.smithi115.stdout:mgr not available, waiting (2/15)... 2024-09-10T19:11:05.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:05.470+0000 7fecec165ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:11:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:11:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: mgrmap e2: y(active, starting, since 0.00273149s) 2024-09-10T19:11:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: Manager daemon y is now available 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' 2024-09-10T19:11:06.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:05 smithi115 ceph-mon[28968]: from='mgr.14100 v1:172.21.15.115:0/1330376133' entity='mgr.y' 2024-09-10T19:11:07.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:07 smithi115 ceph-mon[28968]: mgrmap e3: y(active, since 1.00663s) 2024-09-10T19:11:08.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:08 smithi115 ceph-mon[28968]: mgrmap e4: y(active, since 2s) 2024-09-10T19:11:10.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/345239645' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsid": "4f14fa52-6fa8-11ef-bcea-c7b262605968", 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 0 2024-09-10T19:11:16.555 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:16.556 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T19:11:16.557 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T19:11:16.557 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T19:11:16.557 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T19:11:16.557 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T19:11:16.557 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T19:11:16.560 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.560 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T19:11:16.560 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T19:11:16.560 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T19:11:16.560 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T19:11:16.561 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ], 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T19:10:54.634387+0000", 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T19:11:16.562 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout }, 2024-09-10T19:11:16.563 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T19:11:16.563 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:16.563 INFO:teuthology.orchestra.run.smithi115.stdout:mgr is available 2024-09-10T19:11:18.646 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2140989448' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T19:11:18.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:18.765 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout fsid = 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T19:11:18.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T19:11:18.767 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling cephadm module... 2024-09-10T19:11:19.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2512276782' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T19:11:19.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setuser ceph since I am not root 2024-09-10T19:11:19.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setgroup ceph since I am not root 2024-09-10T19:11:19.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:19.545+0000 7f406022cec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:11:19.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:19.600+0000 7f406022cec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:19.931 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:19.667+0000 7f406022cec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:11:19.931 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:19.792+0000 7f406022cec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:19.932 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:19.929+0000 7f406022cec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:11:20.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:20 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2512276782' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T19:11:20.587 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:20 smithi115 ceph-mon[28968]: mgrmap e5: y(active, since 13s) 2024-09-10T19:11:20.587 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:20 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:20.194+0000 7f406022cec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:11:20.587 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:20 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:20.254+0000 7f406022cec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:11:20.587 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:20 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:20.422+0000 7f406022cec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:11:21.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:21.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T19:11:21.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T19:11:21.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T19:11:21.143 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T19:11:21.144 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:21.144 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-09-10T19:11:21.144 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 5... 2024-09-10T19:11:21.411 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:21 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:21.283+0000 7f406022cec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:11:21.411 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:21 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:21.351+0000 7f406022cec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:11:21.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:21 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1369614806' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T19:11:22.349 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.058+0000 7f406022cec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:11:22.350 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.169+0000 7f406022cec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:11:22.350 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.289+0000 7f406022cec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:22.350 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.348+0000 7f406022cec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:11:22.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.474+0000 7f406022cec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:11:22.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.540+0000 7f406022cec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:11:22.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.598+0000 7f406022cec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:11:22.997 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.732+0000 7f406022cec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:11:22.997 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.795+0000 7f406022cec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:11:22.997 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.859+0000 7f406022cec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:11:22.997 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.939+0000 7f406022cec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:22.997 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:22.995+0000 7f406022cec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:11:23.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:23.052+0000 7f406022cec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:11:24.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:23.790+0000 7f406022cec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: Active manager daemon y restarted 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: mgrmap e6: y(active, starting, since 0.00291093s) 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: Manager daemon y is now available 2024-09-10T19:11:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:11:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:23 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:11:25.106 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-mon[28968]: Found migration_current of "None". Setting to last migration. 2024-09-10T19:11:25.106 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-mon[28968]: mgrmap e7: y(active, since 1.00483s) 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:24.917+0000 7f401261a640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:11:24] ENGINE Error in HTTPServer.serve 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Traceback (most recent call last): 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._connections.run(self.expiration_interval) 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._run(expiration_interval) 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:11:25.107 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s = self.context.wrap_socket( 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: return self.sslsocket_class._create( 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self.do_handshake() 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._sslobj.do_handshake() 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:11:25.108 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 5 is available 2024-09-10T19:11:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:Setting orchestrator backend to cephadm... 2024-09-10T19:11:26.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='client.14122 v1:172.21.15.115:0/1621803592' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T19:11:26.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:24] ENGINE Bus STARTING 2024-09-10T19:11:26.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='client.14122 v1:172.21.15.115:0/1621803592' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:24] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:24] ENGINE Bus STARTED 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:24] ENGINE Error in HTTPServer.serve 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:11:26.164 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:26.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:25 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:26 smithi115 ceph-mon[28968]: from='client.14130 v1:172.21.15.115:0/4231925311' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:26 smithi115 ceph-mon[28968]: mgrmap e8: y(active, since 2s) 2024-09-10T19:11:27.685 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T19:11:27.685 INFO:teuthology.orchestra.run.smithi115.stdout:Generating ssh key... 2024-09-10T19:11:27.994 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:27 smithi115 ceph-mon[28968]: from='client.14132 v1:172.21.15.115:0/1752179199' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:28.655 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Generating public/private rsa key pair. 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Your identification has been saved in /tmp/tmp4v6fp7b9/key 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Your public key has been saved in /tmp/tmp4v6fp7b9/key.pub 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: The key fingerprint is: 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: SHA256:9wPK0cQmFIe02MZMjxYKdAzTJo/aTYW8FWlSkhWEUeA ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: The key's randomart image is: 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: +---[RSA 3072]----+ 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | .===%&*. | 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | .=XX*B | 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | =E*X = | 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | . +o = | 2024-09-10T19:11:28.656 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | o o S + | 2024-09-10T19:11:28.657 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | . . .. + o | 2024-09-10T19:11:28.657 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | o o | 2024-09-10T19:11:28.657 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | . | 2024-09-10T19:11:28.657 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: | | 2024-09-10T19:11:28.657 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:28 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: +----[SHA256]-----+ 2024-09-10T19:11:29.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:29 smithi115 ceph-mon[28968]: from='client.14134 v1:172.21.15.115:0/2872899200' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:29.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:29 smithi115 ceph-mon[28968]: Generating ssh key... 2024-09-10T19:11:29.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:29 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:29.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:29 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:29.949 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGKX9bm+6N6Bxwjjnos3UvsZsjFyp/ePbAaFqNyBoJvxH54A+h51r4weH3kcu6UsYJrPbmb1ComsWTKVHfPt9rS3xGpEp6rbiFz1hf1VCjixU8IYOcPJEefyq6reyv8BEapkL24iA2MfcN752mbXgJr/VsyAHSBytwO1En4CFkkKSOCNq5fFRMmJqJwuvOaPe346MkXjjcIy1bJ2CpWM9O8xzz3/cTEiciYAAEtjyRfjofAxz+oxr5HhfKXsfgbqfHUYsiRsW5AsB/dyyWz13mowEG1+CZJudThgE/rGXRfs2wS8d0/JkltBkskQ/Kxi61B9gpCYS3jf1xhN+Opaosto8NiNkPxwbqm0ydgG59D3qJN1g1JDZraeP3Og1TCvx1ICrzcrvjJbReJUjhYjpTNfwdu9a2wCfB3be3XCvkV8yRxVzZ54FTME0UoLBlPxu3l8CUEhcs6ESsdQNW3dN5ZdwsX5ADdFQvrGWtlhibe0/g8Nd/xh7XnMbd0fusp18= ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:11:29.950 INFO:teuthology.orchestra.run.smithi115.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:11:29.950 INFO:teuthology.orchestra.run.smithi115.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T19:11:29.955 INFO:teuthology.orchestra.run.smithi115.stdout:Adding host smithi115... 2024-09-10T19:11:30.657 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:30 smithi115 ceph-mon[28968]: from='client.14136 v1:172.21.15.115:0/2968541416' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:31.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:31 smithi115 ceph-mon[28968]: from='client.14138 v1:172.21.15.115:0/3409455732' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "addr": "172.21.15.115", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:31.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:31 smithi115 ceph-mon[28968]: Deploying cephadm binary to smithi115 2024-09-10T19:11:33.766 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Added host 'smithi115' with addr '172.21.15.115' 2024-09-10T19:11:33.766 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mon service... 2024-09-10T19:11:34.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:34 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:34.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:34 smithi115 ceph-mon[28968]: Added host smithi115 2024-09-10T19:11:34.322 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:34 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:35.016 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T19:11:35.016 INFO:teuthology.orchestra.run.smithi115.stdout:Deploying unmanaged mgr service... 2024-09-10T19:11:35.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:35 smithi115 ceph-mon[28968]: from='client.14140 v1:172.21.15.115:0/310576680' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:35.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:35 smithi115 ceph-mon[28968]: Saving service mon spec with placement count:5 2024-09-10T19:11:35.517 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:35 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:36.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:36 smithi115 ceph-mon[28968]: from='client.14142 v1:172.21.15.115:0/2180989012' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:36.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:36 smithi115 ceph-mon[28968]: Saving service mgr spec with placement count:2 2024-09-10T19:11:36.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:36 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:36.951 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:36 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:36.953 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T19:11:38.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:38 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:38.723 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:38 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3236948795' entity='client.admin' 2024-09-10T19:11:39.967 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:39 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3634141908' entity='client.admin' 2024-09-10T19:11:39.967 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:39 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:39.969 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling the dashboard module... 2024-09-10T19:11:41.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-mon[28968]: from='mgr.14118 v1:172.21.15.115:0/3469485467' entity='mgr.y' 2024-09-10T19:11:41.155 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2537831986' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T19:11:41.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setuser ceph since I am not root 2024-09-10T19:11:41.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setgroup ceph since I am not root 2024-09-10T19:11:41.156 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.152+0000 7fb6a79e7ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:11:41.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.208+0000 7fb6a79e7ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:41.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.272+0000 7fb6a79e7ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:11:41.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.397+0000 7fb6a79e7ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:41.795 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.533+0000 7fb6a79e7ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:11:41.795 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.793+0000 7fb6a79e7ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:11:42.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:42 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2537831986' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T19:11:42.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:42 smithi115 ceph-mon[28968]: mgrmap e9: y(active, since 17s) 2024-09-10T19:11:42.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:41.854+0000 7fb6a79e7ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:11:42.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:42.018+0000 7fb6a79e7ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:11:43.145 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:43 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3353084112' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T19:11:43.145 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:42.879+0000 7fb6a79e7ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:11:43.145 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:42.943+0000 7fb6a79e7ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:11:43.149 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:43.152 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T19:11:43.152 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T19:11:43.153 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T19:11:43.153 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T19:11:43.153 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:43.153 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for the mgr to restart... 2024-09-10T19:11:43.153 INFO:teuthology.orchestra.run.smithi115.stdout:Waiting for mgr epoch 9... 2024-09-10T19:11:43.772 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:43.660+0000 7fb6a79e7ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:11:44.073 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:43.770+0000 7fb6a79e7ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:11:44.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:43.889+0000 7fb6a79e7ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:11:44.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:43.947+0000 7fb6a79e7ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:11:44.074 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.071+0000 7fb6a79e7ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:11:44.329 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.141+0000 7fb6a79e7ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:11:44.330 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.201+0000 7fb6a79e7ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:11:44.620 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.339+0000 7fb6a79e7ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:11:44.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.405+0000 7fb6a79e7ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:11:44.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.472+0000 7fb6a79e7ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:11:44.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.558+0000 7fb6a79e7ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:11:44.621 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.618+0000 7fb6a79e7ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:11:44.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:44.676+0000 7fb6a79e7ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:45.433+0000 7fb6a79e7ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: Active manager daemon y restarted 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: mgrmap e10: y(active, starting, since 0.00316593s) 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:11:45.583 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: Manager daemon y is now available 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:11:45.584 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:46.766+0000 7fb658544640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:11:46] ENGINE Error in HTTPServer.serve 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Traceback (most recent call last): 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._connections.run(self.expiration_interval) 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._run(expiration_interval) 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:11:46.888 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s = self.context.wrap_socket( 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: return self.sslsocket_class._create( 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self.do_handshake() 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._sslobj.do_handshake() 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:11:46.889 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:11:46 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout { 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout } 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:mgr epoch 9 is available 2024-09-10T19:11:47.166 INFO:teuthology.orchestra.run.smithi115.stdout:Generating a dashboard self-signed certificate... 2024-09-10T19:11:47.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: mgrmap e11: y(active, since 1.00524s) 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:46] ENGINE Bus STARTING 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:46] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:46] ENGINE Bus STARTED 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: [10/Sep/2024:19:11:46] ENGINE Error in HTTPServer.serve 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:11:47.808 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:11:47.809 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:11:47.810 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:47 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:48.406 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T19:11:48.406 INFO:teuthology.orchestra.run.smithi115.stdout:Creating initial admin user... 2024-09-10T19:11:48.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:48 smithi115 ceph-mon[28968]: from='client.14162 v1:172.21.15.115:0/4275567524' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:48.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:48 smithi115 ceph-mon[28968]: mgrmap e12: y(active, since 2s) 2024-09-10T19:11:48.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:48.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:48.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:50.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:49 smithi115 ceph-mon[28968]: from='client.14164 v1:172.21.15.115:0/3818978216' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:11:50.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:49 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:50.073 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6.F6sSIc061beiyOMvkEgOrInpRXBRqWiqq8nXYdZIhvH8TdspDli", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725995509, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T19:11:50.074 INFO:teuthology.orchestra.run.smithi115.stdout:Fetching dashboard port number... 2024-09-10T19:11:50.901 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:50 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2136965086' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T19:11:51.553 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T19:11:51.560 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T19:11:51.560 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout disabled 2024-09-10T19:11:51.568 INFO:teuthology.orchestra.run.smithi115.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T19:11:51.568 INFO:teuthology.orchestra.run.smithi115.stdout:systemctl: stdout inactive 2024-09-10T19:11:51.568 INFO:teuthology.orchestra.run.smithi115.stdout:firewalld.service is not enabled 2024-09-10T19:11:51.568 INFO:teuthology.orchestra.run.smithi115.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T19:11:51.570 INFO:teuthology.orchestra.run.smithi115.stdout:Ceph Dashboard is now available at: 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: URL: https://smithi115.front.sepia.ceph.com:8443/ 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: User: admin 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: Password: unsh3at10w 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout:Saving cluster configuration to /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config directory 2024-09-10T19:11:51.571 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling autotune for osd_memory_target 2024-09-10T19:11:52.973 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:52 smithi115 ceph-mon[28968]: mgrmap e13: y(active, since 6s) 2024-09-10T19:11:53.879 INFO:teuthology.orchestra.run.smithi115.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T19:11:53.879 INFO:teuthology.orchestra.run.smithi115.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T19:11:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /sbin/cephadm shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:11:53.880 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.880 INFO:teuthology.orchestra.run.smithi115.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: sudo /sbin/cephadm shell 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.881 INFO:teuthology.orchestra.run.smithi115.stdout: ceph telemetry on 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout:For more information see: 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:11:53.882 INFO:teuthology.orchestra.run.smithi115.stdout:Bootstrap complete. 2024-09-10T19:11:53.915 INFO:tasks.cephadm:Fetching config... 2024-09-10T19:11:53.915 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:11:53.916 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T19:11:53.934 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T19:11:53.934 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:11:53.934 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T19:11:53.990 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T19:11:53.990 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:11:53.990 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T19:11:54.055 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T19:11:54.055 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:11:54.055 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T19:11:54.110 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T19:11:54.111 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGKX9bm+6N6Bxwjjnos3UvsZsjFyp/ePbAaFqNyBoJvxH54A+h51r4weH3kcu6UsYJrPbmb1ComsWTKVHfPt9rS3xGpEp6rbiFz1hf1VCjixU8IYOcPJEefyq6reyv8BEapkL24iA2MfcN752mbXgJr/VsyAHSBytwO1En4CFkkKSOCNq5fFRMmJqJwuvOaPe346MkXjjcIy1bJ2CpWM9O8xzz3/cTEiciYAAEtjyRfjofAxz+oxr5HhfKXsfgbqfHUYsiRsW5AsB/dyyWz13mowEG1+CZJudThgE/rGXRfs2wS8d0/JkltBkskQ/Kxi61B9gpCYS3jf1xhN+Opaosto8NiNkPxwbqm0ydgG59D3qJN1g1JDZraeP3Og1TCvx1ICrzcrvjJbReJUjhYjpTNfwdu9a2wCfB3be3XCvkV8yRxVzZ54FTME0UoLBlPxu3l8CUEhcs6ESsdQNW3dN5ZdwsX5ADdFQvrGWtlhibe0/g8Nd/xh7XnMbd0fusp18= ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T19:11:54.195 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGKX9bm+6N6Bxwjjnos3UvsZsjFyp/ePbAaFqNyBoJvxH54A+h51r4weH3kcu6UsYJrPbmb1ComsWTKVHfPt9rS3xGpEp6rbiFz1hf1VCjixU8IYOcPJEefyq6reyv8BEapkL24iA2MfcN752mbXgJr/VsyAHSBytwO1En4CFkkKSOCNq5fFRMmJqJwuvOaPe346MkXjjcIy1bJ2CpWM9O8xzz3/cTEiciYAAEtjyRfjofAxz+oxr5HhfKXsfgbqfHUYsiRsW5AsB/dyyWz13mowEG1+CZJudThgE/rGXRfs2wS8d0/JkltBkskQ/Kxi61B9gpCYS3jf1xhN+Opaosto8NiNkPxwbqm0ydgG59D3qJN1g1JDZraeP3Og1TCvx1ICrzcrvjJbReJUjhYjpTNfwdu9a2wCfB3be3XCvkV8yRxVzZ54FTME0UoLBlPxu3l8CUEhcs6ESsdQNW3dN5ZdwsX5ADdFQvrGWtlhibe0/g8Nd/xh7XnMbd0fusp18= ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:11:54.206 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGKX9bm+6N6Bxwjjnos3UvsZsjFyp/ePbAaFqNyBoJvxH54A+h51r4weH3kcu6UsYJrPbmb1ComsWTKVHfPt9rS3xGpEp6rbiFz1hf1VCjixU8IYOcPJEefyq6reyv8BEapkL24iA2MfcN752mbXgJr/VsyAHSBytwO1En4CFkkKSOCNq5fFRMmJqJwuvOaPe346MkXjjcIy1bJ2CpWM9O8xzz3/cTEiciYAAEtjyRfjofAxz+oxr5HhfKXsfgbqfHUYsiRsW5AsB/dyyWz13mowEG1+CZJudThgE/rGXRfs2wS8d0/JkltBkskQ/Kxi61B9gpCYS3jf1xhN+Opaosto8NiNkPxwbqm0ydgG59D3qJN1g1JDZraeP3Og1TCvx1ICrzcrvjJbReJUjhYjpTNfwdu9a2wCfB3be3XCvkV8yRxVzZ54FTME0UoLBlPxu3l8CUEhcs6ESsdQNW3dN5ZdwsX5ADdFQvrGWtlhibe0/g8Nd/xh7XnMbd0fusp18= ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T19:11:54.340 INFO:teuthology.orchestra.run.smithi150.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGKX9bm+6N6Bxwjjnos3UvsZsjFyp/ePbAaFqNyBoJvxH54A+h51r4weH3kcu6UsYJrPbmb1ComsWTKVHfPt9rS3xGpEp6rbiFz1hf1VCjixU8IYOcPJEefyq6reyv8BEapkL24iA2MfcN752mbXgJr/VsyAHSBytwO1En4CFkkKSOCNq5fFRMmJqJwuvOaPe346MkXjjcIy1bJ2CpWM9O8xzz3/cTEiciYAAEtjyRfjofAxz+oxr5HhfKXsfgbqfHUYsiRsW5AsB/dyyWz13mowEG1+CZJudThgE/rGXRfs2wS8d0/JkltBkskQ/Kxi61B9gpCYS3jf1xhN+Opaosto8NiNkPxwbqm0ydgG59D3qJN1g1JDZraeP3Og1TCvx1ICrzcrvjJbReJUjhYjpTNfwdu9a2wCfB3be3XCvkV8yRxVzZ54FTME0UoLBlPxu3l8CUEhcs6ESsdQNW3dN5ZdwsX5ADdFQvrGWtlhibe0/g8Nd/xh7XnMbd0fusp18= ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:11:54.353 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T19:11:54.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:54 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2444688355' entity='client.admin' 2024-09-10T19:11:54.601 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:11:57.776 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T19:11:57.776 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T19:11:58.347 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:11:58.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2646643965' entity='client.admin' 2024-09-10T19:11:59.653 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:59.653 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:59.653 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:11:59.653 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:11:59.653 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:59.654 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:11:59 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:00.877 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi150 2024-09-10T19:12:00.878 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:12:00.878 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.conf 2024-09-10T19:12:00.897 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:12:00.897 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:00.955 INFO:tasks.cephadm:Adding host smithi150 to orchestrator... 2024-09-10T19:12:00.955 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch host add smithi150 2024-09-10T19:12:01.194 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:01.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='client.14174 v1:172.21.15.115:0/1265399411' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:01.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:01 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:03 smithi115 ceph-mon[28968]: from='client.14176 v1:172.21.15.115:0/844202093' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi150", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:04.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:04 smithi115 ceph-mon[28968]: Deploying cephadm binary to smithi150 2024-09-10T19:12:05.511 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi150' with addr '172.21.15.150' 2024-09-10T19:12:06.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T19:12:06.429 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:06.793 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:06.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:06.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: Added host smithi150 2024-09-10T19:12:06.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:06.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:06.794 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:06 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:07.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:07 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:08.118 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:12:08.119 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.115", "hostname": "smithi115", "labels": [], "status": ""}, {"addr": "172.21.15.150", "hostname": "smithi150", "labels": [], "status": ""}] 2024-09-10T19:12:08.669 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T19:12:08.669 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T19:12:08.903 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:09.024 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:08 smithi115 ceph-mon[28968]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:10.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='client.14178 v1:172.21.15.115:0/549913843' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:12:10.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:10.150 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:10.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:10.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:10.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi150", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:12:10.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:10.151 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:09 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:10.873 INFO:teuthology.orchestra.run.smithi115.stderr:adjusted tunables profile to default 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2704245750' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:11.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:11.466 INFO:tasks.cephadm:Adding mon.a on smithi115 2024-09-10T19:12:11.466 INFO:tasks.cephadm:Adding mon.c on smithi115 2024-09-10T19:12:11.466 INFO:tasks.cephadm:Adding mon.b on smithi150 2024-09-10T19:12:11.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi115:[v1:172.21.15.115:6789]=a;smithi115:[v1:172.21.15.115:6790]=c;smithi150:[v1:172.21.15.150:6789]=b' 2024-09-10T19:12:11.716 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:11 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:11 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:12:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:11 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2704245750' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T19:12:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:11 smithi115 ceph-mon[28968]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:12 smithi115 ceph-mon[28968]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:13.295 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mon update... 2024-09-10T19:12:13.903 DEBUG:teuthology.orchestra.run.smithi115:mon.c> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T19:12:13.906 DEBUG:teuthology.orchestra.run.smithi150:mon.b> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T19:12:13.908 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T19:12:13.909 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='client.14182 v1:172.21.15.150:0/3618106100' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:[v1:172.21.15.115:6789]=a;smithi115:[v1:172.21.15.115:6790]=c;smithi150:[v1:172.21.15.150:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: Saving service mon spec with placement smithi115:[v1:172.21.15.115:6789]=a;smithi115:[v1:172.21.15.115:6790]=c;smithi150:[v1:172.21.15.150:6789]=b;count:3 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:14.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:14.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:14.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:14.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:14.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: Deploying daemon mon.b on smithi150 2024-09-10T19:12:14.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:14 smithi115 ceph-mon[28968]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:14.746 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:16 smithi115 ceph-mon[28968]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:18.045 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:12:18.045 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":1,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","modified":"2024-09-10T19:10:51.842870Z","created":"2024-09-10T19:10:51.842870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T19:12:18.045 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 1 2024-09-10T19:12:18.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:18 smithi150 systemd[1]: Started Ceph mon.b for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:12:18.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:18 smithi115 ceph-mon[28968]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:18.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/2873037498' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:19.623 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T19:12:19.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T19:12:19.887 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:12:24.007 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:12:24.007 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":2,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","modified":"2024-09-10T19:12:18.759941Z","created":"2024-09-10T19:10:51.842870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T19:12:24.007 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 2 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: mon.a calling monitor election 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: mon.b calling monitor election 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: monmap e2: 2 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: fsmap 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: mgrmap e13: y(active, since 38s) 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: overall HEALTH_OK 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:24.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: mon.a calling monitor election 2024-09-10T19:12:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: mon.b calling monitor election 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: monmap e2: 2 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: fsmap 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: mgrmap e13: y(active, since 38s) 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: overall HEALTH_OK 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:24.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:25.122 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:24 smithi115 ceph-mon[28968]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:25.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:24 smithi115 ceph-mon[28968]: Deploying daemon mon.c on smithi115 2024-09-10T19:12:25.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:24 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/4282018355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:25.123 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:24 smithi150 ceph-mon[33096]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:24 smithi150 ceph-mon[33096]: Deploying daemon mon.c on smithi115 2024-09-10T19:12:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:24 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/4282018355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:25.578 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T19:12:25.579 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T19:12:25.817 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:12:25.969 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:12:25 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:12:25.759+0000 7fb68bf79640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T19:12:26.641 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 podman[39493]: 2024-09-10 19:12:26.578627342 +0000 UTC m=+0.121304580 container create 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:12:26.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[28968]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 podman[39493]: 2024-09-10 19:12:26.798194723 +0000 UTC m=+0.340871965 container init 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:12:26.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 podman[39493]: 2024-09-10 19:12:26.80248427 +0000 UTC m=+0.345161520 container start 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pidfile_write: ignore empty --pid-file 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: load: jerasure load: lrc 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: RocksDB version: 6.15.5 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Compile date Sep 9 2024 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: DB SUMMARY 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: DB Session ID: R93MOFBAYEY839M5QYY6 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: CURRENT file: CURRENT 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T19:12:26.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.error_if_exists: 0 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.create_if_missing: 0 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.paranoid_checks: 1 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.env: 0x559c541d9440 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.fs: Posix File System 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.info_log: 0x559c56354e80 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T19:12:26.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.statistics: (nil) 2024-09-10T19:12:26.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.use_fsync: 0 2024-09-10T19:12:26.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_log_file_size: 0 2024-09-10T19:12:26.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T19:12:26.916 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_fallocate: 1 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.use_direct_reads: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.db_log_dir: 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T19:12:26.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.write_buffer_manager: 0x559c56412840 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T19:12:26.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.unordered_write: 0 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T19:12:26.919 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.row_cache: None 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.wal_filter: None 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.preserve_deletes: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.two_write_queues: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.atomic_flush: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.log_readahead_size: 0 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T19:12:26.920 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_background_jobs: 2 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_background_compactions: -1 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_subcompactions: 1 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T19:12:26.921 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_open_files: -1 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_background_flushes: -1 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Compression algorithms supported: 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kZSTD supported: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kXpressCompression supported: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kBZip2Compression supported: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kLZ4Compression supported: 1 2024-09-10T19:12:26.922 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kZlibCompression supported: 1 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: kSnappyCompression supported: 1 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.merge_operator: 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_filter: None 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T19:12:26.923 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559c562fba38) 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks: 1 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: pin_top_level_index_and_filter: 1 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_type: 0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_index_type: 0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_shortening: 1 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: hash_index_allow_collision: 1 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: checksum: 1 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: no_block_cache: 0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache: 0x559c563311f0 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_name: BinnedLRUCache 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_options: 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: capacity : 536870912 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: num_shard_bits : 4 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: strict_capacity_limit : 0 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: high_pri_pool_ratio: 0.000 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_cache_compressed: (nil) 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: persistent_cache: (nil) 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size: 4096 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_size_deviation: 10 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_restart_interval: 16 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: index_block_restart_interval: 1 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: metadata_block_size: 4096 2024-09-10T19:12:26.925 INFO:journalctl@ceph.mon.c.smithi115.stdout: partition_filters: 0 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: use_delta_encoding: 1 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: whole_key_filtering: 1 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: verify_compression: 0 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: read_amp_bytes_per_bit: 0 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: format_version: 4 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_index_compression: 1 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout: block_align: 0 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression: NoCompression 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T19:12:26.926 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.num_levels: 7 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T19:12:26.927 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T19:12:26.928 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T19:12:26.929 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T19:12:26.930 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T19:12:26.930 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T19:12:26.931 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.inplace_update_support: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.bloom_locality: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.max_successive_merges: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T19:12:26.932 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.ttl: 2592000 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.enable_blob_files: false 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.min_blob_size: 0 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T19:12:26.933 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995546826312, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995546826617, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1626, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725995546, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "6ced8595-631c-4f12-8eee-d24b24e887b6", "db_session_id": "R93MOFBAYEY839M5QYY6"}} 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725995546826925, "job": 1, "event": "recovery_finished"} 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x559c56364700 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: DB pointer 0x559c56334000 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T19:12:26.934 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** DB Stats ** 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:12:26.935 INFO:journalctl@ceph.mon.c.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T19:12:26.936 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.937 INFO:journalctl@ceph.mon.c.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T19:12:26.938 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.938 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** Compaction Stats [default] ** 2024-09-10T19:12:26.938 INFO:journalctl@ceph.mon.c.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T19:12:26.938 INFO:journalctl@ceph.mon.c.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T19:12:26.938 INFO:journalctl@ceph.mon.c.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T19:12:26.939 INFO:journalctl@ceph.mon.c.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T19:12:26.939 INFO:journalctl@ceph.mon.c.smithi115.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T19:12:26.939 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T19:12:26.940 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: using public_addrv v1:172.21.15.115:6790/0 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: starting mon.c rank -1 at public addrs v1:172.21.15.115:6790/0 at bind addrs v1:172.21.15.115:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(???) e0 preinit fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout: e1 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T19:12:26.941 INFO:journalctl@ceph.mon.c.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout: legacy client fscid: -1 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout: 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout: No filesystems configured 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T19:12:26.942 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='client.14182 v1:172.21.15.150:0/3618106100' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi115:[v1:172.21.15.115:6789]=a;smithi115:[v1:172.21.15.115:6790]=c;smithi150:[v1:172.21.15.150:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: Saving service mon spec with placement smithi115:[v1:172.21.15.115:6789]=a;smithi115:[v1:172.21.15.115:6790]=c;smithi150:[v1:172.21.15.150:6789]=b;count:3 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:26.943 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: Deploying daemon mon.b on smithi150 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/2873037498' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.a calling monitor election 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.944 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.b calling monitor election 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: monmap e2: 2 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: fsmap 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mgrmap e13: y(active, since 38s) 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: overall HEALTH_OK 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:26.945 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: Deploying daemon mon.c on smithi115 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/4282018355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 ceph-mon[39520]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 bash[39493]: 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 2024-09-10T19:12:26.946 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:26 smithi115 systemd[1]: Started Ceph mon.c for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:12:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:26 smithi150 ceph-mon[33096]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:32.107 INFO:teuthology.orchestra.run.smithi150.stdout: 2024-09-10T19:12:32.107 INFO:teuthology.orchestra.run.smithi150.stdout:{"epoch":3,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","modified":"2024-09-10T19:12:26.850489Z","created":"2024-09-10T19:10:51.842870Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6789","nonce":0}]},"addr":"172.21.15.150:6789/0","public_addr":"172.21.15.150:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6790","nonce":0}]},"addr":"172.21.15.115:6790/0","public_addr":"172.21.15.115:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T19:12:32.108 INFO:teuthology.orchestra.run.smithi150.stderr:dumped monmap epoch 3 2024-09-10T19:12:32.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: mon.b calling monitor election 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: mon.a calling monitor election 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:32.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: fsmap 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: mgrmap e13: y(active, since 46s) 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: mon.c (rank 2) addr v1:172.21.15.115:6790/0 is down (out of quorum) 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: mon.b calling monitor election 2024-09-10T19:12:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: mon.a calling monitor election 2024-09-10T19:12:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: fsmap 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: mgrmap e13: y(active, since 46s) 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: mon.c (rank 2) addr v1:172.21.15.115:6790/0 is down (out of quorum) 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:32.733 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T19:12:32.733 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T19:12:33.015 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:33.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:32 smithi150 ceph-mon[33096]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:33.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:32 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:33.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:32 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/2916171866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:33.036 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:32 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:33.261 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:32 smithi115 ceph-mon[28968]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:33.261 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:32 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:33.261 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:32 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/2916171866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:33.261 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:32 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: mon.b calling monitor election 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: mon.a calling monitor election 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.121 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: fsmap 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: mgrmap e13: y(active, since 46s) 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T19:12:34.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: mon.c (rank 2) addr v1:172.21.15.115:6790/0 is down (out of quorum) 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/2916171866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T19:12:34.123 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:34.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mon.c calling monitor election 2024-09-10T19:12:34.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mon.c calling monitor election 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mon.b calling monitor election 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mon.a calling monitor election 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: fsmap 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: mgrmap e13: y(active, since 48s) 2024-09-10T19:12:34.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: Cluster is now healthy 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: overall HEALTH_OK 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:34.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mon.c calling monitor election 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mon.c calling monitor election 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mon.b calling monitor election 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mon.a calling monitor election 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: fsmap 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: mgrmap e13: y(active, since 48s) 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T19:12:34.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: Cluster is now healthy 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: overall HEALTH_OK 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:34.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:34.751 INFO:teuthology.orchestra.run.smithi115.stdout:# minimal ceph.conf for 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:12:34.751 INFO:teuthology.orchestra.run.smithi115.stdout:[global] 2024-09-10T19:12:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: fsid = 4f14fa52-6fa8-11ef-bcea-c7b262605968 2024-09-10T19:12:34.751 INFO:teuthology.orchestra.run.smithi115.stdout: mon_host = 172.21.15.115:6789/0 172.21.15.150:6789/0 v1:172.21.15.115:6790/0 2024-09-10T19:12:34.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mon.c calling monitor election 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mon.c calling monitor election 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mon.b calling monitor election 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mon.a calling monitor election 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: monmap e3: 3 mons at {a=v1:172.21.15.115:6789/0,b=v1:172.21.15.150:6789/0,c=v1:172.21.15.115:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T19:12:34.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: fsmap 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: mgrmap e13: y(active, since 48s) 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: Cluster is now healthy 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: overall HEALTH_OK 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:34.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:34.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:34.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:35.301 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T19:12:35.301 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:12:35.301 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T19:12:35.328 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:12:35.328 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:35.393 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:12:35.393 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T19:12:35.426 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:12:35.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:35.497 INFO:tasks.cephadm:Adding mgr.y on smithi115 2024-09-10T19:12:35.497 INFO:tasks.cephadm:Adding mgr.x on smithi150 2024-09-10T19:12:35.497 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi115=y;smithi150=x' 2024-09-10T19:12:35.776 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:12:35.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[39520]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T19:12:35.873 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[39520]: Reconfiguring daemon mon.a on smithi115 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/253418804' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[28968]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[28968]: Reconfiguring daemon mon.a on smithi115 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/253418804' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:35.874 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:35 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:35 smithi150 ceph-mon[33096]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T19:12:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:35 smithi150 ceph-mon[33096]: Reconfiguring daemon mon.a on smithi115 2024-09-10T19:12:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:35 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/253418804' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:35 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:12:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:35 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.852 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:36.852 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.852 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:36.853 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:36 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:36.861 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:36 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:37.334 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled mgr update... 2024-09-10T19:12:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: Reconfiguring mon.c (monmap changed)... 2024-09-10T19:12:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: Reconfiguring daemon mon.c on smithi115 2024-09-10T19:12:37.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:37.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: Reconfiguring mon.c (monmap changed)... 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: Reconfiguring daemon mon.c on smithi115 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:37.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: Reconfiguring mon.c (monmap changed)... 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: Reconfiguring daemon mon.c on smithi115 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T19:12:37.918 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:37.941 DEBUG:teuthology.orchestra.run.smithi150:mgr.x> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T19:12:37.943 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T19:12:37.943 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:12:37.943 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T19:12:37.960 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T19:12:37.960 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 19:12:26.290861466 +0000 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 19:07:50.284196315 +0000 2024-09-10T19:12:38.016 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 19:07:50.284196315 +0000 2024-09-10T19:12:38.017 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 19:07:50.284196315 +0000 2024-09-10T19:12:38.017 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T19:12:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T19:12:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T19:12:38.080 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000233894 s, 2.2 MB/s 2024-09-10T19:12:38.082 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T19:12:38.138 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.193 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 19:12:26.291861446 +0000 2024-09-10T19:12:38.194 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 19:07:50.285196295 +0000 2024-09-10T19:12:38.194 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 19:07:50.285196295 +0000 2024-09-10T19:12:38.194 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 19:07:50.285196295 +0000 2024-09-10T19:12:38.194 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T19:12:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T19:12:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T19:12:38.256 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000220601 s, 2.3 MB/s 2024-09-10T19:12:38.257 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T19:12:38.314 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 2024-09-10T19:12:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T19:12:38.369 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 19:12:26.291861446 +0000 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.370 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.370 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T19:12:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T19:12:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T19:12:38.433 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000232198 s, 2.2 MB/s 2024-09-10T19:12:38.434 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T19:12:38.491 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 2024-09-10T19:12:38.547 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T19:12:38.547 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2024-09-10 19:12:26.291861446 +0000 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.548 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: 2024-09-10 19:07:50.283196334 +0000 2024-09-10T19:12:38.548 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T19:12:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2024-09-10T19:12:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2024-09-10T19:12:38.612 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000175723 s, 2.9 MB/s 2024-09-10T19:12:38.613 DEBUG:teuthology.orchestra.run.smithi115:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T19:12:38.670 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:12:38.670 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T19:12:38.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T19:12:38.686 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 19:12:16.926241583 +0000 2024-09-10T19:12:38.742 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.743 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.743 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T19:12:38.806 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T19:12:38.807 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T19:12:38.807 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000291619 s, 1.8 MB/s 2024-09-10T19:12:38.808 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T19:12:38.864 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2024-09-10T19:12:38.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:38 smithi150 ceph-mon[33096]: from='client.24103 v1:172.21.15.150:0/2716523970' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:38.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:38 smithi150 ceph-mon[33096]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T19:12:38.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:38 smithi150 ceph-mon[33096]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:38.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:38 smithi150 ceph-mon[33096]: Reconfiguring mon.b (monmap changed)... 2024-09-10T19:12:38.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:38 smithi150 ceph-mon[33096]: Reconfiguring daemon mon.b on smithi150 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 19:12:16.927241563 +0000 2024-09-10T19:12:38.898 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.899 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.899 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:38.899 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T19:12:38.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[28968]: from='client.24103 v1:172.21.15.150:0/2716523970' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:38.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[28968]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T19:12:38.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[28968]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:38.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[28968]: Reconfiguring mon.b (monmap changed)... 2024-09-10T19:12:38.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[28968]: Reconfiguring daemon mon.b on smithi150 2024-09-10T19:12:38.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[39520]: from='client.24103 v1:172.21.15.150:0/2716523970' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi115=y;smithi150=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:38.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[39520]: Saving service mgr spec with placement smithi115=y;smithi150=x;count:2 2024-09-10T19:12:38.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[39520]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:38.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[39520]: Reconfiguring mon.b (monmap changed)... 2024-09-10T19:12:38.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:38 smithi115 ceph-mon[39520]: Reconfiguring daemon mon.b on smithi150 2024-09-10T19:12:38.962 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T19:12:38.962 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T19:12:38.962 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000188212 s, 2.7 MB/s 2024-09-10T19:12:38.963 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T19:12:39.021 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2024-09-10T19:12:39.076 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 19:12:16.927241563 +0000 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 19:07:50.512362297 +0000 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 19:07:50.512362297 +0000 2024-09-10T19:12:39.077 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 19:07:50.512362297 +0000 2024-09-10T19:12:39.077 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T19:12:39.142 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T19:12:39.142 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T19:12:39.142 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000224773 s, 2.3 MB/s 2024-09-10T19:12:39.143 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T19:12:39.201 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2024-09-10T19:12:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T19:12:39.257 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T19:12:39.257 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T19:12:39.257 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T19:12:39.258 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T19:12:39.258 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2024-09-10 19:12:16.927241563 +0000 2024-09-10T19:12:39.258 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:39.258 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:39.258 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: 2024-09-10 19:07:50.511362245 +0000 2024-09-10T19:12:39.258 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T19:12:39.323 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2024-09-10T19:12:39.323 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2024-09-10T19:12:39.323 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000451313 s, 1.1 MB/s 2024-09-10T19:12:39.325 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T19:12:39.382 INFO:tasks.cephadm:Deploying osd.0 on smithi115 with /dev/vg_nvme/lv_4... 2024-09-10T19:12:39.383 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T19:12:39.619 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:40.372 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.372 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.372 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.373 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:40 smithi150 ceph-mon[33096]: Deploying daemon mgr.x on smithi150 2024-09-10T19:12:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[28968]: Deploying daemon mgr.x on smithi150 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:40.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:40 smithi115 ceph-mon[39520]: Deploying daemon mgr.x on smithi150 2024-09-10T19:12:41.062 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:40 smithi150 podman[34919]: 2024-09-10 19:12:40.972592422 +0000 UTC m=+0.130051050 container create 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD) 2024-09-10T19:12:41.291 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T19:12:41.291 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:41 smithi115 ceph-mon[28968]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:41.292 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:41 smithi115 ceph-mon[39520]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:41.357 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-mon[33096]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:41.357 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 podman[34919]: 2024-09-10 19:12:41.183530665 +0000 UTC m=+0.340989292 container init 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T19:12:41.357 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 podman[34919]: 2024-09-10 19:12:41.186314894 +0000 UTC m=+0.343773530 container start 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:12:41.358 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.300+0000 7f6689fbdec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:12:41.358 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.355+0000 7f6689fbdec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:12:41.383 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T19:12:41.397 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T19:12:41.397 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T19:12:41.397 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115727 s, 906 MB/s 2024-09-10T19:12:41.397 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T19:12:41.632 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 bash[34919]: 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 2024-09-10T19:12:41.633 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 systemd[1]: Started Ceph mgr.x for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:12:41.633 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.426+0000 7f6689fbdec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:12:41.633 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.557+0000 7f6689fbdec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:12:41.977 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2024-09-10T19:12:41.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.702+0000 7f6689fbdec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:12:41.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:41 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:41.984+0000 7f6689fbdec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:12:42.213 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:12:42.247 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:42.044+0000 7f6689fbdec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:12:42.247 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:42.215+0000 7f6689fbdec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:12:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:42.657 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.657 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:42.658 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:43.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:43.082+0000 7f6689fbdec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:12:43.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:43.147+0000 7f6689fbdec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:12:43.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:43.807 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[28968]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:43.808 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.809 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T19:12:43.809 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:43.809 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:43.809 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.809 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:43 smithi115 ceph-mon[39520]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T19:12:43.870 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:12:43.871 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:12:43.871 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:43.871 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-mon[33096]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:12:44.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:43.868+0000 7f6689fbdec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:12:44.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:43.979+0000 7f6689fbdec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:12:44.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.099+0000 7f6689fbdec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:12:44.411 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.159+0000 7f6689fbdec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:12:44.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.285+0000 7f6689fbdec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:12:44.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.352+0000 7f6689fbdec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:12:44.412 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.409+0000 7f6689fbdec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-mon[33096]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-mon[33096]: from='client.14217 v1:172.21.15.115:0/2902489049' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.542+0000 7f6689fbdec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.606+0000 7f6689fbdec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:12:44.674 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.672+0000 7f6689fbdec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:12:44.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[39520]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:44.886 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[39520]: from='client.14217 v1:172.21.15.115:0/2902489049' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[28968]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[28968]: from='client.14217 v1:172.21.15.115:0/2902489049' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:12:44.887 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:45.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.753+0000 7f6689fbdec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:12:45.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.812+0000 7f6689fbdec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:12:45.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:44.872+0000 7f6689fbdec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:12:45.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:12:45.622+0000 7f6689fbdec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:12:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: Standby manager daemon x started 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:12:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:45 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:12:45.964 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.964 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.964 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:45.964 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: Standby manager daemon x started 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:12:45.965 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:12:45.966 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:12:45.966 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:12:45.966 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.966 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.966 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: Standby manager daemon x started 2024-09-10T19:12:45.967 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:12:45.968 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:12:45.968 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:12:45.968 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:45 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/2666890455' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:12:47.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]: dispatch 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]': finished 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T19:12:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:46 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:12:47.152 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]: dispatch 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]': finished 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]: dispatch 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2521290683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "004c62f5-ed21-4e2e-aeb6-41ff66d018e2"}]': finished 2024-09-10T19:12:47.153 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T19:12:47.154 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:46 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:12:48.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:47 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1106632285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:12:48.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:47 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1106632285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:12:48.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:47 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1106632285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:12:49.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:48 smithi150 ceph-mon[33096]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:48 smithi115 ceph-mon[28968]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:49.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:48 smithi115 ceph-mon[39520]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:51.108 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:50 smithi115 ceph-mon[28968]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:51.108 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:50 smithi115 ceph-mon[39520]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:51.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:50 smithi150 ceph-mon[33096]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[28968]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[39520]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T19:12:52.985 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:53.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:52 smithi150 ceph-mon[33096]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T19:12:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:53.862 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:53 smithi115 ceph-mon[28968]: Deploying daemon osd.0 on smithi115 2024-09-10T19:12:53.863 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:53 smithi115 ceph-mon[39520]: Deploying daemon osd.0 on smithi115 2024-09-10T19:12:54.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:53 smithi150 ceph-mon[33096]: Deploying daemon osd.0 on smithi115 2024-09-10T19:12:54.998 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:54 smithi115 ceph-mon[28968]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:54.998 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:54 smithi115 ceph-mon[39520]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:55.047 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:54 smithi150 ceph-mon[33096]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:56.888 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:56 smithi115 ceph-mon[28968]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:56.889 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:56 smithi115 ceph-mon[39520]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:56 smithi150 ceph-mon[33096]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:57.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:58.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:58.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:12:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:12:58 smithi115 ceph-mon[28968]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:12:58 smithi115 ceph-mon[39520]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:12:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:12:58 smithi150 ceph-mon[33096]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:00.049 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 0 on host 'smithi115' 2024-09-10T19:13:00.626 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.626 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[28968]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.627 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:00 smithi115 ceph-mon[39520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:00 smithi150 ceph-mon[33096]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T19:13:00.649 DEBUG:teuthology.orchestra.run.smithi115:osd.0> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T19:13:00.651 INFO:tasks.cephadm:Deploying osd.1 on smithi115 with /dev/vg_nvme/lv_3... 2024-09-10T19:13:00.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T19:13:02.259 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:02.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[28968]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T19:13:02.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[28968]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T19:13:02.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[28968]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[28968]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[39520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[39520]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[39520]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:02.382 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-mon[39520]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:02.383 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0[43377]: 2024-09-10T19:13:02.053+0000 7f866f0d2640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T19:13:02.383 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:02 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0[43377]: 2024-09-10T19:13:02.055+0000 7f8669f09640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:13:02.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:02 smithi150 ceph-mon[33096]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T19:13:02.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:02 smithi150 ceph-mon[33096]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T19:13:02.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:02 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:02.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:02 smithi150 ceph-mon[33096]: from='osd.0 v1:172.21.15.115:6801/1864670008' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:02.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:02 smithi150 ceph-mon[33096]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:03 smithi150 ceph-mon[33096]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:03 smithi150 ceph-mon[33096]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:03 smithi150 ceph-mon[33096]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T19:13:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:03 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:03 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[28968]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[28968]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[28968]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T19:13:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:03.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:03.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[39520]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:03.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[39520]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:03.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[39520]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T19:13:03.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:03.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:03 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:13:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:13:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: osd.0 v1:172.21.15.115:6801/1864670008 boot 2024-09-10T19:13:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:13:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: osd.0 v1:172.21.15.115:6801/1864670008 boot 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:04.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: osd.0 v1:172.21.15.115:6801/1864670008 boot 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:04.553 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T19:13:04.638 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T19:13:04.651 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T19:13:04.652 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T19:13:04.652 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124241 s, 844 MB/s 2024-09-10T19:13:04.652 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T19:13:05.211 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2024-09-10T19:13:05.233 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[28968]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi115 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[28968]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[39520]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi115 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T19:13:05.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:05 smithi115 ceph-mon[39520]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T19:13:05.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:05 smithi150 ceph-mon[33096]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T19:13:05.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:05 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi115 2024-09-10T19:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:05 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 16064M 2024-09-10T19:13:05.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:05 smithi150 ceph-mon[33096]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T19:13:05.449 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[28968]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[39520]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:07.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:07.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:07.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:07 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:07 smithi150 ceph-mon[33096]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:07 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:07 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:07.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:07 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:08.353 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:08 smithi115 ceph-mon[39520]: from='client.34139 v1:172.21.15.115:0/482308299' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:08.353 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:08 smithi115 ceph-mon[28968]: from='client.34139 v1:172.21.15.115:0/482308299' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:08.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:08 smithi150 ceph-mon[33096]: from='client.34139 v1:172.21.15.115:0/482308299' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:09.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:09 smithi150 ceph-mon[33096]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:09.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:09 smithi115 ceph-mon[28968]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:09.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:09 smithi115 ceph-mon[39520]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:10.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]: dispatch 2024-09-10T19:13:10.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]': finished 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/632568868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]: dispatch 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]': finished 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:10.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/632568868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:10.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]: dispatch 2024-09-10T19:13:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/940874493' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55469ced-c422-4c09-8962-39641a099a5e"}]': finished 2024-09-10T19:13:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T19:13:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:10 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/632568868' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:12.628 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:12 smithi115 ceph-mon[28968]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:12.628 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:12 smithi115 ceph-mon[39520]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:12.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:12 smithi150 ceph-mon[33096]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:14.875 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:14 smithi115 ceph-mon[39520]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:14.875 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:14 smithi115 ceph-mon[28968]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:14.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:14 smithi150 ceph-mon[33096]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:15.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T19:13:15.741 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:15.742 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T19:13:15.742 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T19:13:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:16.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:16 smithi115 ceph-mon[39520]: Deploying daemon osd.1 on smithi115 2024-09-10T19:13:16.734 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:16 smithi115 ceph-mon[39520]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:16.734 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:16 smithi115 ceph-mon[28968]: Deploying daemon osd.1 on smithi115 2024-09-10T19:13:16.734 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:16 smithi115 ceph-mon[28968]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:16 smithi150 ceph-mon[33096]: Deploying daemon osd.1 on smithi115 2024-09-10T19:13:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:16 smithi150 ceph-mon[33096]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:18.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:18 smithi150 ceph-mon[33096]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:18.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:18 smithi115 ceph-mon[28968]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:18 smithi115 ceph-mon[39520]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:20.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:20 smithi150 ceph-mon[33096]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:20.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:20.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:20.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[28968]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[39520]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:20.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:20.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:22.707 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:22 smithi115 ceph-mon[28968]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:22.708 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:22 smithi115 ceph-mon[39520]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:22.742 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 1 on host 'smithi115' 2024-09-10T19:13:22.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:22 smithi150 ceph-mon[33096]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:23.344 DEBUG:teuthology.orchestra.run.smithi115:osd.1> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T19:13:23.386 INFO:tasks.cephadm:Deploying osd.2 on smithi115 with /dev/vg_nvme/lv_2... 2024-09-10T19:13:23.387 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T19:13:23.672 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.076 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.077 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.077 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.077 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.077 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.077 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:23 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T19:13:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:23 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:24.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:24.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[28968]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:24.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-mon[39520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:24.915 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1[48960]: 2024-09-10T19:13:24.755+0000 7f58c2a91640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T19:13:24.915 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1[48960]: 2024-09-10T19:13:24.758+0000 7f58be0c9640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: from='osd.1 v1:172.21.15.115:6805/1865971090' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:25.034 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:24 smithi150 ceph-mon[33096]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:25.836 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[39520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:25.836 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[39520]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T19:13:25.837 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[28968]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[28968]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:25.840 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:25 smithi150 ceph-mon[33096]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:25 smithi150 ceph-mon[33096]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T19:13:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:26.346 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T19:13:26.437 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T19:13:26.455 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T19:13:26.455 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T19:13:26.455 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0154051 s, 681 MB/s 2024-09-10T19:13:26.455 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: osd.1 v1:172.21.15.115:6805/1865971090 boot 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:27.078 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: osd.1 v1:172.21.15.115:6805/1865971090 boot 2024-09-10T19:13:27.079 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T19:13:27.079 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:26 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:27.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2024-09-10T19:13:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:13:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:13:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T19:13:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: osd.1 v1:172.21.15.115:6805/1865971090 boot 2024-09-10T19:13:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T19:13:27.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:26 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:13:27.676 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:28.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T19:13:28.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.111 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.112 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.113 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:28.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:29.075 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[28968]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:29.075 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi115 2024-09-10T19:13:29.075 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T19:13:29.076 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[39520]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:29.076 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi115 2024-09-10T19:13:29.076 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:28 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T19:13:29.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:28 smithi150 ceph-mon[33096]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:29.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:28 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi115 2024-09-10T19:13:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:28 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 8032M 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:29.987 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:30.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:30.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:31.018 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:30 smithi115 ceph-mon[28968]: from='client.14271 v1:172.21.15.115:0/1065626826' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:31.019 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:30 smithi115 ceph-mon[28968]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:31.019 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:30 smithi115 ceph-mon[39520]: from='client.14271 v1:172.21.15.115:0/1065626826' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:31.019 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:30 smithi115 ceph-mon[39520]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:31.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:30 smithi150 ceph-mon[33096]: from='client.14271 v1:172.21.15.115:0/1065626826' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:31.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:30 smithi150 ceph-mon[33096]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:32.011 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]: dispatch 2024-09-10T19:13:32.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]': finished 2024-09-10T19:13:32.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[39520]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T19:13:32.012 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:32.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]: dispatch 2024-09-10T19:13:32.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]': finished 2024-09-10T19:13:32.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[28968]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T19:13:32.013 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:31 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:32.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:31 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]: dispatch 2024-09-10T19:13:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:31 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/910342937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5dc996f6-4d04-4b76-a7a1-333bdf21b1cc"}]': finished 2024-09-10T19:13:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:31 smithi150 ceph-mon[33096]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T19:13:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:31 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:33.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:32 smithi150 ceph-mon[33096]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:32 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1044379178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:33.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:32 smithi115 ceph-mon[28968]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:33.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:32 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1044379178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:33.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:32 smithi115 ceph-mon[39520]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:33.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:32 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1044379178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:35.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:34 smithi150 ceph-mon[33096]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:34 smithi115 ceph-mon[28968]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:35.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:34 smithi115 ceph-mon[39520]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:37.094 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:36 smithi115 ceph-mon[28968]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:37.094 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:36 smithi115 ceph-mon[39520]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:37.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:36 smithi150 ceph-mon[33096]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T19:13:38.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:37 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:38.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T19:13:38.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:37 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:38.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T19:13:38.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:37 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:39.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:38 smithi115 ceph-mon[39520]: Deploying daemon osd.2 on smithi115 2024-09-10T19:13:39.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:38 smithi115 ceph-mon[39520]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:39.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:38 smithi115 ceph-mon[28968]: Deploying daemon osd.2 on smithi115 2024-09-10T19:13:39.028 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:38 smithi115 ceph-mon[28968]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:39.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:38 smithi150 ceph-mon[33096]: Deploying daemon osd.2 on smithi115 2024-09-10T19:13:39.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:38 smithi150 ceph-mon[33096]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:41.072 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:40 smithi115 ceph-mon[28968]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:41.072 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:40 smithi115 ceph-mon[39520]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:41.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:40 smithi150 ceph-mon[33096]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:42.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[28968]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:42.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:42.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:42.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:42.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[39520]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:42.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:42.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:42.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:43.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:42 smithi150 ceph-mon[33096]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:43.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:43.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:43.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.040 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 2 on host 'smithi115' 2024-09-10T19:13:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.627 DEBUG:teuthology.orchestra.run.smithi115:osd.2> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T19:13:45.629 INFO:tasks.cephadm:Deploying osd.3 on smithi115 with /dev/vg_nvme/lv_1... 2024-09-10T19:13:45.629 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T19:13:45.904 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[39520]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[39520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[28968]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:45.905 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:45 smithi115 ceph-mon[28968]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:13:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:45 smithi150 ceph-mon[33096]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:46.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:45 smithi150 ceph-mon[33096]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T19:13:46.708 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:47.235 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[39520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:47.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-mon[28968]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:47.236 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2[54777]: 2024-09-10T19:13:47.047+0000 7f6d301fc640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T19:13:47.236 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:47 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2[54777]: 2024-09-10T19:13:47.050+0000 7f6d2902f640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:13:47.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:47.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T19:13:47.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T19:13:47.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:47.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: from='osd.2 v1:172.21.15.115:6809/5537849' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:47.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:47 smithi150 ceph-mon[33096]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:13:48.313 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:13:48.313 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:13:48.313 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:48.313 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T19:13:48.313 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.314 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:48 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:48.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:48 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:49.152 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[28968]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[28968]: osd.2 v1:172.21.15.115:6809/5537849 boot 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[28968]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[39520]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[39520]: osd.2 v1:172.21.15.115:6809/5537849 boot 2024-09-10T19:13:49.160 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[39520]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T19:13:49.161 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:49 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:49 smithi150 ceph-mon[33096]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T19:13:49.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:49 smithi150 ceph-mon[33096]: osd.2 v1:172.21.15.115:6809/5537849 boot 2024-09-10T19:13:49.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:49 smithi150 ceph-mon[33096]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T19:13:49.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:49 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:13:49.522 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T19:13:49.626 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T19:13:49.640 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2024-09-10T19:13:49.641 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2024-09-10T19:13:49.641 INFO:teuthology.orchestra.run.smithi115.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124622 s, 841 MB/s 2024-09-10T19:13:49.641 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.282 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.283 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:50 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T19:13:50.304 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi115:vg_nvme/lv_1 2024-09-10T19:13:50.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:50.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:50 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T19:13:50.543 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:13:51.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi115 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: pgmap v71: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:13:51.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi115 2024-09-10T19:13:51.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T19:13:51.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: pgmap v71: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:51.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T19:13:51.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T19:13:51.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:51 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:13:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi115 2024-09-10T19:13:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 5354M 2024-09-10T19:13:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: pgmap v71: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T19:13:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T19:13:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:51 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:13:51.913 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:51 smithi115 sudo[58680]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:51.913 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:51 smithi115 sudo[58680]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:51.913 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:51 smithi115 sudo[58680]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:51.913 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:51 smithi115 sudo[58680]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:52.226 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58704]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:52.226 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58704]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:52.226 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58704]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:52.226 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58704]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:52.596 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58776]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:52.596 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58776]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:52.596 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58776]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:52.596 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58776]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:52.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:13:52.596 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:52.597 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:52.598 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:52.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T19:13:52.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T19:13:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:13:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:13:52.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:13:52.853 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58831]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:52.853 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58831]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:52.853 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58831]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:52.854 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58831]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:53.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58851]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:53.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58851]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:53.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58851]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:53.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:52 smithi115 sudo[58851]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:53.444 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58855]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:53.445 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58855]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:53.445 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58855]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:53.445 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58855]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:53.446 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58858]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:53.446 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58858]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:53.446 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58858]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:53.446 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58858]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:53.697 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58883]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:53.697 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58883]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:53.697 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58883]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:53.697 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 sudo[58883]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: from='client.14304 v1:172.21.15.115:0/1988999162' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 sudo[35576]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 sudo[35576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 sudo[35576]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:53.956 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:53 smithi150 sudo[35576]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:54.089 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: from='client.14304 v1:172.21.15.115:0/1988999162' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:54.089 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: from='client.14304 v1:172.21.15.115:0/1988999162' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:54.090 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:53 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:54.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 sudo[35579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 sudo[35579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 sudo[35579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:54.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 sudo[35579]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:54.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58934]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T19:13:54.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58934]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:54.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58934]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:54.723 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58934]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:55.011 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:55.011 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.012 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58937]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58937]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58937]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 sudo[58937]: pam_unix(sudo:session): session closed for user root 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.013 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T19:13:55.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:55.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.014 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:13:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:13:55.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:55.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1539599440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:55.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='client.34205 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:55.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='client.34205 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]': finished 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3501551432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1539599440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='client.34205 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='client.34205 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]': finished 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:13:55.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:55 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3501551432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1539599440' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='client.34205 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]: dispatch 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='client.34205 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0"}]': finished 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:13:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:55 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3501551432' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:13:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:56 smithi150 ceph-mon[33096]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:56 smithi115 ceph-mon[28968]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:56 smithi115 ceph-mon[39520]: pgmap v78: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:13:58.767 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:13:58 smithi115 ceph-mon[28968]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 249 KiB/s wr, 17 op/s 2024-09-10T19:13:58.768 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:13:58 smithi115 ceph-mon[39520]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 249 KiB/s wr, 17 op/s 2024-09-10T19:13:59.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:13:58 smithi150 ceph-mon[33096]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.2 KiB/s rd, 249 KiB/s wr, 17 op/s 2024-09-10T19:14:00.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[39520]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.7 KiB/s rd, 181 KiB/s wr, 12 op/s 2024-09-10T19:14:00.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T19:14:00.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:00.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[28968]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.7 KiB/s rd, 181 KiB/s wr, 12 op/s 2024-09-10T19:14:00.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T19:14:00.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:01.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:00 smithi150 ceph-mon[33096]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.7 KiB/s rd, 181 KiB/s wr, 12 op/s 2024-09-10T19:14:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T19:14:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:02.057 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:01 smithi115 ceph-mon[28968]: Deploying daemon osd.3 on smithi115 2024-09-10T19:14:02.057 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:01 smithi115 ceph-mon[39520]: Deploying daemon osd.3 on smithi115 2024-09-10T19:14:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:01 smithi150 ceph-mon[33096]: Deploying daemon osd.3 on smithi115 2024-09-10T19:14:03.000 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:02 smithi115 ceph-mon[39520]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 141 KiB/s wr, 9 op/s 2024-09-10T19:14:03.000 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:02 smithi115 ceph-mon[28968]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 141 KiB/s wr, 9 op/s 2024-09-10T19:14:03.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:02 smithi150 ceph-mon[33096]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.9 KiB/s rd, 141 KiB/s wr, 9 op/s 2024-09-10T19:14:04.999 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:04 smithi115 ceph-mon[28968]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-10T19:14:04.999 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:04 smithi115 ceph-mon[39520]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-10T19:14:05.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:04 smithi150 ceph-mon[33096]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-10T19:14:05.878 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:05.878 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:05.878 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:05.878 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:05.879 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:05.879 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:05 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:06.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:05 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:06.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:05 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:06.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:05 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:07.122 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:06 smithi115 ceph-mon[28968]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 122 KiB/s wr, 8 op/s 2024-09-10T19:14:07.122 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:06 smithi115 ceph-mon[39520]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 122 KiB/s wr, 8 op/s 2024-09-10T19:14:07.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:06 smithi150 ceph-mon[33096]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.5 KiB/s rd, 122 KiB/s wr, 8 op/s 2024-09-10T19:14:08.462 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 3 on host 'smithi115' 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T19:14:09.023 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T19:14:09.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.024 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:08 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.025 DEBUG:teuthology.orchestra.run.smithi115:osd.3> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T19:14:09.027 INFO:tasks.cephadm:Deploying osd.4 on smithi150 with /dev/vg_nvme/lv_4... 2024-09-10T19:14:09.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.056 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:08 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:09.282 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:14:10.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:10 smithi150 ceph-mon[33096]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T19:14:10.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:10 smithi150 ceph-mon[33096]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T19:14:10.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:10.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:10 smithi150 ceph-mon[33096]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:14:10.297 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:10 smithi150 ceph-mon[33096]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:14:10.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[28968]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T19:14:10.371 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[28968]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[28968]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[28968]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[39520]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[39520]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[39520]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2024-09-10T19:14:10.372 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-mon[39520]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T19:14:10.373 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3[60613]: 2024-09-10T19:14:10.114+0000 7fa796931640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T19:14:10.373 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:14:10 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3[60613]: 2024-09-10T19:14:10.118+0000 7fa78ff65640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:14:10.957 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T19:14:11.045 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T19:14:11.058 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T19:14:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T19:14:11.058 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116362 s, 901 MB/s 2024-09-10T19:14:11.058 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T19:14:11.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:11 smithi150 ceph-mon[33096]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:14:11.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:11 smithi150 ceph-mon[33096]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T19:14:11.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:11 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.344 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:11 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[28968]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[28968]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[39520]: from='osd.3 v1:172.21.15.115:6813/3633830068' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[39520]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.365 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:11 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:11.740 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_4 2024-09-10T19:14:12.032 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:14:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: osd.3 v1:172.21.15.115:6813/3633830068 boot 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi115 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:12 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:14:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:14:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: osd.3 v1:172.21.15.115:6813/3633830068 boot 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi115 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:14:12.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: osd.3 v1:172.21.15.115:6813/3633830068 boot 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi115 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:12.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 4016M 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:12.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:12 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:13.328 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:13 smithi150 ceph-mon[33096]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T19:14:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:13 smithi115 ceph-mon[28968]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T19:14:13.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:13 smithi115 ceph-mon[39520]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T19:14:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:14 smithi150 ceph-mon[33096]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:14 smithi150 ceph-mon[33096]: from='client.24232 v1:172.21.15.150:0/1161162923' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:14 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:14 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:14.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:14 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[28968]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[28968]: from='client.24232 v1:172.21.15.150:0/1161162923' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[39520]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[39520]: from='client.24232 v1:172.21.15.150:0/1161162923' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:14 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/3598926009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]': finished 2024-09-10T19:14:16.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T19:14:16.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:16 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/3598926009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]': finished 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/3598926009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]: dispatch 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c"}]': finished 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T19:14:16.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:16 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:17.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:17 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/2918569618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:17.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:17 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/2918569618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:17.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:17 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/2918569618' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:18.775 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:18 smithi150 ceph-mon[33096]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:18.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:18 smithi115 ceph-mon[28968]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:18 smithi115 ceph-mon[39520]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:20.804 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:20 smithi150 ceph-mon[33096]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:20.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:20 smithi115 ceph-mon[28968]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:20.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:20 smithi115 ceph-mon[39520]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:22.665 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:22 smithi150 ceph-mon[33096]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:22 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T19:14:22.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:22 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:22.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[28968]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:22.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T19:14:22.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:22.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[39520]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:22.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T19:14:22.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:22 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:23.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:23 smithi150 ceph-mon[33096]: Deploying daemon osd.4 on smithi150 2024-09-10T19:14:23.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:23 smithi115 ceph-mon[28968]: Deploying daemon osd.4 on smithi150 2024-09-10T19:14:23.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:23 smithi115 ceph-mon[39520]: Deploying daemon osd.4 on smithi150 2024-09-10T19:14:24.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:24 smithi150 ceph-mon[33096]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:24.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:24 smithi115 ceph-mon[28968]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:24.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:24 smithi115 ceph-mon[39520]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:26.608 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:26 smithi150 ceph-mon[33096]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:26.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:26 smithi115 ceph-mon[28968]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:26.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:26 smithi115 ceph-mon[39520]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:27.605 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:27.605 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:27.605 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:27.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:27.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:27.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:27.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:27.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:27.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:29.230 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:29 smithi150 ceph-mon[33096]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:29.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:29 smithi115 ceph-mon[28968]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:29 smithi115 ceph-mon[39520]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:30.253 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 4 on host 'smithi150' 2024-09-10T19:14:30.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:30.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:30 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:30.936 DEBUG:teuthology.orchestra.run.smithi150:osd.4> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T19:14:30.939 INFO:tasks.cephadm:Deploying osd.5 on smithi150 with /dev/vg_nvme/lv_3... 2024-09-10T19:14:30.939 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T19:14:31.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:31.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:31.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T19:14:31.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:31.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:30 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:32.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-mon[33096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T19:14:32.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-mon[33096]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T19:14:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-mon[33096]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:32.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-mon[33096]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:32.133 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:14:32.030+0000 7f4f91cdb640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T19:14:32.133 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:14:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:14:32.034+0000 7f4f8cb12640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:14:32.259 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:14:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[28968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T19:14:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[28968]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T19:14:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:32.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[28968]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[28968]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[39520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[39520]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[39520]: from='osd.4 v1:172.21.15.150:6800/2544371837' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:32.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:32 smithi115 ceph-mon[39520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:33 smithi150 ceph-mon[33096]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:33 smithi150 ceph-mon[33096]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:33 smithi150 ceph-mon[33096]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T19:14:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:33 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:33.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:33 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[28968]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[28968]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[28968]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T19:14:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:33.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[39520]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T19:14:33.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[39520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:33.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[39520]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T19:14:33.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:33.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: osd.4 v1:172.21.15.150:6800/2544371837 boot 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:34.309 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.310 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:34 smithi150 ceph-mon[33096]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: osd.4 v1:172.21.15.150:6800/2544371837 boot 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[28968]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: osd.4 v1:172.21.15.150:6800/2544371837 boot 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T19:14:34.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:34.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:34 smithi115 ceph-mon[39520]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T19:14:34.907 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T19:14:34.998 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T19:14:35.012 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T19:14:35.012 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T19:14:35.012 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124456 s, 843 MB/s 2024-09-10T19:14:35.012 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T19:14:35.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:35 smithi150 ceph-mon[33096]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:35.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:35 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi150 2024-09-10T19:14:35.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:35 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T19:14:35.306 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:35 smithi150 ceph-mon[33096]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T19:14:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[28968]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi150 2024-09-10T19:14:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T19:14:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[28968]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T19:14:35.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[39520]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:35.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi150 2024-09-10T19:14:35.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 17088M 2024-09-10T19:14:35.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:35 smithi115 ceph-mon[39520]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T19:14:35.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_3 2024-09-10T19:14:35.840 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:14:37.199 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:37 smithi150 ceph-mon[33096]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:37 smithi115 ceph-mon[28968]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:37.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:37 smithi115 ceph-mon[39520]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:38.090 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:38.091 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:38.091 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:38.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:38.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:38.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:38.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:14:38.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:14:38.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:39.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:39 smithi150 ceph-mon[33096]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-10T19:14:39.181 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:39 smithi150 ceph-mon[33096]: from='client.24259 v1:172.21.15.150:0/650673646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:39 smithi115 ceph-mon[28968]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-10T19:14:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:39 smithi115 ceph-mon[28968]: from='client.24259 v1:172.21.15.150:0/650673646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:39.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:39 smithi115 ceph-mon[39520]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-09-10T19:14:39.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:39 smithi115 ceph-mon[39520]: from='client.24259 v1:172.21.15.150:0/650673646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:14:40.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/2402415328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:40 smithi150 ceph-mon[33096]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:40 smithi150 ceph-mon[33096]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]': finished 2024-09-10T19:14:40.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:40 smithi150 ceph-mon[33096]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T19:14:40.380 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:40 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/2402415328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[28968]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[28968]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]': finished 2024-09-10T19:14:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[28968]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T19:14:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/2402415328' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[39520]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]: dispatch 2024-09-10T19:14:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[39520]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4265f75c-402b-4094-b27f-dd1854d8dffe"}]': finished 2024-09-10T19:14:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[39520]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T19:14:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:40 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:41 smithi150 ceph-mon[33096]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T19:14:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:41 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/484470064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:41 smithi115 ceph-mon[28968]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T19:14:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:41 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/484470064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:41 smithi115 ceph-mon[39520]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T19:14:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:41 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/484470064' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:14:43.206 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:43 smithi150 ceph-mon[33096]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T19:14:43.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:43 smithi115 ceph-mon[28968]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T19:14:43.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:43 smithi115 ceph-mon[39520]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T19:14:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:45 smithi150 ceph-mon[33096]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T19:14:45.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:45 smithi115 ceph-mon[28968]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T19:14:45.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:45 smithi115 ceph-mon[39520]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T19:14:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:46 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T19:14:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:46 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:46 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T19:14:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:46 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:46 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T19:14:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:46 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:47.250 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:47 smithi150 ceph-mon[33096]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:14:47.250 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:47 smithi150 ceph-mon[33096]: Deploying daemon osd.5 on smithi150 2024-09-10T19:14:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:47 smithi115 ceph-mon[28968]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:14:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:47 smithi115 ceph-mon[28968]: Deploying daemon osd.5 on smithi150 2024-09-10T19:14:47.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:47 smithi115 ceph-mon[39520]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:14:47.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:47 smithi115 ceph-mon[39520]: Deploying daemon osd.5 on smithi150 2024-09-10T19:14:49.326 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:49 smithi150 ceph-mon[33096]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:49.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:49 smithi115 ceph-mon[28968]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:49.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:49 smithi115 ceph-mon[39520]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:51 smithi150 ceph-mon[33096]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:51 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:51 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:51 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[28968]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[39520]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:51.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:51.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:51.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:51 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:53.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:53 smithi150 ceph-mon[33096]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:53.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:53 smithi115 ceph-mon[28968]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:53.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:53 smithi115 ceph-mon[39520]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:54.212 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 5 on host 'smithi150' 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.820 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:54.821 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.821 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:54 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.844 DEBUG:teuthology.orchestra.run.smithi150:osd.5> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T19:14:54.846 INFO:tasks.cephadm:Deploying osd.6 on smithi150 with /dev/vg_nvme/lv_2... 2024-09-10T19:14:54.846 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T19:14:54.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:54.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:54 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:55.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-mon[33096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T19:14:55.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-mon[33096]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T19:14:55.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:55.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-mon[33096]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:55.860 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-mon[33096]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:55.861 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:14:55.525+0000 7fef7765c640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T19:14:55.861 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:14:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:14:55.530+0000 7fef74497640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[28968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[28968]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[28968]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[28968]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[39520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[39520]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[39520]: from='osd.5 v1:172.21.15.150:6804/2087992752' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:55.917 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:55 smithi115 ceph-mon[39520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:14:56.661 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:14:56.731 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:56 smithi150 ceph-mon[33096]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:56.731 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:56 smithi150 ceph-mon[33096]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:56.731 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:56 smithi150 ceph-mon[33096]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T19:14:56.732 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:56 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:56.732 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:56 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:56.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[28968]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:56.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[28968]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[28968]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[39520]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[39520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[39520]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:56.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:56 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:57.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:57 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:14:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:57 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:14:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:57 smithi150 ceph-mon[33096]: osd.5 v1:172.21.15.150:6804/2087992752 boot 2024-09-10T19:14:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:57 smithi150 ceph-mon[33096]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T19:14:57.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:14:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:14:57.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[28968]: osd.5 v1:172.21.15.150:6804/2087992752 boot 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[28968]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[39520]: osd.5 v1:172.21.15.150:6804/2087992752 boot 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[39520]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T19:14:57.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi150 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.666 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:58.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:58.667 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:58 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi150 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:58.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi150 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 8544M 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:14:58.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:58 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:14:59.034 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T19:14:59.114 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T19:14:59.127 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T19:14:59.127 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T19:14:59.127 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114976 s, 912 MB/s 2024-09-10T19:14:59.127 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T19:14:59.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_2 2024-09-10T19:14:59.798 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:14:59 smithi150 ceph-mon[33096]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T19:14:59.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:14:59 smithi115 ceph-mon[28968]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T19:14:59.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:14:59 smithi115 ceph-mon[39520]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T19:15:00.017 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:00.819 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:00 smithi150 ceph-mon[33096]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:00.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:00 smithi115 ceph-mon[28968]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:00.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:00 smithi115 ceph-mon[39520]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:02 smithi150 ceph-mon[33096]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:02 smithi150 ceph-mon[33096]: from='client.24286 v1:172.21.15.150:0/2372375518' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:02 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:02 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:02.805 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:02 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:02.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[28968]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:02.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[28968]: from='client.24286 v1:172.21.15.150:0/2372375518' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[39520]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[39520]: from='client.24286 v1:172.21.15.150:0/2372375518' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:02.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:02 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:04.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/639086906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]': finished 2024-09-10T19:15:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T19:15:04.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:04.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:04.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/639086906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]': finished 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/639086906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]: dispatch 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7023616-9190-4517-9b6b-63c8112c6f92"}]': finished 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T19:15:04.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:05.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:05 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/1062815726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:05.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:05 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/1062815726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:05.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:05 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/1062815726' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:06.823 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:06 smithi150 ceph-mon[33096]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:06.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:06 smithi115 ceph-mon[28968]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:06.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:06 smithi115 ceph-mon[39520]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:08.710 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:08 smithi150 ceph-mon[33096]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:08.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:08 smithi115 ceph-mon[28968]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:08.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:08 smithi115 ceph-mon[39520]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:10 smithi150 ceph-mon[33096]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:15:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:10 smithi150 ceph-mon[33096]: Deploying daemon osd.6 on smithi150 2024-09-10T19:15:10.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[28968]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:10.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:15:10.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:10.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[28968]: Deploying daemon osd.6 on smithi150 2024-09-10T19:15:10.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[39520]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:10.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T19:15:10.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:10.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:10 smithi115 ceph-mon[39520]: Deploying daemon osd.6 on smithi150 2024-09-10T19:15:12.717 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:12 smithi150 ceph-mon[33096]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:12.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:12 smithi115 ceph-mon[28968]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:12.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:12 smithi115 ceph-mon[39520]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:14.859 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:14 smithi150 ceph-mon[33096]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:14.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:14 smithi115 ceph-mon[39520]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:14.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:14 smithi115 ceph-mon[28968]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:15.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:15.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:15.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:17.222 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:16 smithi150 ceph-mon[33096]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:17.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:16 smithi115 ceph-mon[39520]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:17.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:16 smithi115 ceph-mon[28968]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:18.296 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 6 on host 'smithi150' 2024-09-10T19:15:18.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:18.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:18 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.910 DEBUG:teuthology.orchestra.run.smithi150:osd.6> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T19:15:18.915 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:18.915 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.915 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:18 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:18.953 INFO:tasks.cephadm:Deploying osd.7 on smithi150 with /dev/vg_nvme/lv_1... 2024-09-10T19:15:18.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T19:15:19.682 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:20.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-mon[33096]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:15:20.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-mon[33096]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:15:20.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:20.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-mon[33096]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:20.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-mon[33096]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:20.133 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:15:19.930+0000 7f6c613ba640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T19:15:20.133 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:15:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:15:19.936+0000 7f6c5e1f5640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:15:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[28968]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:15:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[28968]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:15:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[28968]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[28968]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:20.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[39520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T19:15:20.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[39520]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T19:15:20.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:20.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[39520]: from='osd.6 v1:172.21.15.150:6808/1869693771' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:20.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:19 smithi115 ceph-mon[39520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:15:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: osd.6 v1:172.21.15.150:6808/1869693771 boot 2024-09-10T19:15:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:15:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:20 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: osd.6 v1:172.21.15.150:6808/1869693771 boot 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: osd.6 v1:172.21.15.150:6808/1869693771 boot 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T19:15:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:20 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:15:21.755 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T19:15:21.841 INFO:teuthology.orchestra.run.smithi150.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T19:15:21.855 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10+0 records in 2024-09-10T19:15:21.855 INFO:teuthology.orchestra.run.smithi150.stderr:10+0 records out 2024-09-10T19:15:21.855 INFO:teuthology.orchestra.run.smithi150.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124325 s, 843 MB/s 2024-09-10T19:15:21.856 INFO:teuthology.orchestra.run.smithi150.stderr:--> Zapping successful for: 2024-09-10T19:15:22.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:21 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:15:22.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:21 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:15:22.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:21 smithi150 ceph-mon[33096]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:15:22.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:15:22.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:15:22.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[28968]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:15:22.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:15:22.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:15:22.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:21 smithi115 ceph-mon[39520]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T19:15:22.483 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi150:vg_nvme/lv_1 2024-09-10T19:15:22.813 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:23.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:22 smithi150 ceph-mon[33096]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:23.213 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:22 smithi150 ceph-mon[33096]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:15:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:22 smithi115 ceph-mon[28968]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:22 smithi115 ceph-mon[28968]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:15:23.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:22 smithi115 ceph-mon[39520]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:23.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:22 smithi115 ceph-mon[39520]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T19:15:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:24 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:25.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:25.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:25.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:25.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:25.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:24 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:25 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi150 2024-09-10T19:15:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:25 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T19:15:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:26.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:25 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:26.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi150 2024-09-10T19:15:26.460 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi150 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 5696M 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T19:15:26.461 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:25 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:27.316 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:26 smithi150 ceph-mon[33096]: from='client.24313 v1:172.21.15.150:0/275325623' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:27.316 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:26 smithi150 ceph-mon[33096]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:26 smithi115 ceph-mon[28968]: from='client.24313 v1:172.21.15.150:0/275325623' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:26 smithi115 ceph-mon[28968]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:27.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:26 smithi115 ceph-mon[39520]: from='client.24313 v1:172.21.15.150:0/275325623' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi150:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:27.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:26 smithi115 ceph-mon[39520]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:28.128 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:27 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/3234313542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.128 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:27 smithi150 ceph-mon[33096]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.128 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:27 smithi150 ceph-mon[33096]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]': finished 2024-09-10T19:15:28.128 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:27 smithi150 ceph-mon[33096]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:15:28.128 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:27 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:28.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/3234313542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[28968]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[28968]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]': finished 2024-09-10T19:15:28.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[28968]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:15:28.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:28.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/3234313542' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[39520]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]: dispatch 2024-09-10T19:15:28.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[39520]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "044ab88d-b207-4094-aa68-ac7908dcea54"}]': finished 2024-09-10T19:15:28.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[39520]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T19:15:28.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:27 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:29.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:28 smithi150 ceph-mon[33096]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:29.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:28 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/421587786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:29.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:28 smithi115 ceph-mon[28968]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:29.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:28 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/421587786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:28 smithi115 ceph-mon[39520]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:28 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/421587786' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T19:15:31.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:30 smithi150 ceph-mon[33096]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:30 smithi115 ceph-mon[28968]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:31.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:30 smithi115 ceph-mon[39520]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:33.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:32 smithi150 ceph-mon[33096]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:32 smithi115 ceph-mon[28968]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:33.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:32 smithi115 ceph-mon[39520]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:34.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:33 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:15:34.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:33 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:34.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:33 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:15:34.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:33 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:34.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T19:15:34.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:33 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:35.008 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:34 smithi150 ceph-mon[33096]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:35.008 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:34 smithi150 ceph-mon[33096]: Deploying daemon osd.7 on smithi150 2024-09-10T19:15:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:34 smithi115 ceph-mon[28968]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:34 smithi115 ceph-mon[28968]: Deploying daemon osd.7 on smithi150 2024-09-10T19:15:35.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:34 smithi115 ceph-mon[39520]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:35.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:34 smithi115 ceph-mon[39520]: Deploying daemon osd.7 on smithi150 2024-09-10T19:15:37.194 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:36 smithi150 ceph-mon[33096]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:36 smithi115 ceph-mon[28968]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:37.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:36 smithi115 ceph-mon[39520]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:39.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:38 smithi150 ceph-mon[33096]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:39.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:39.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:39.267 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:38 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[28968]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:39.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[39520]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:39.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:39.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:39.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:38 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:41.080 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:40 smithi150 ceph-mon[33096]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:40 smithi115 ceph-mon[28968]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:41.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:40 smithi115 ceph-mon[39520]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:42.097 INFO:teuthology.orchestra.run.smithi150.stdout:Created osd(s) 7 on host 'smithi150' 2024-09-10T19:15:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:42 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T19:15:42.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:42.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:42 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:42.754 DEBUG:teuthology.orchestra.run.smithi150:osd.7> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T19:15:42.756 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T19:15:42.757 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T19:15:43.008 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:15:43.718 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[28968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:15:43.718 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[28968]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:15:43.903 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-mon[33096]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:15:43.904 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-mon[33096]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:15:43.904 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:43.904 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-mon[33096]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:43.904 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-mon[33096]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:43.904 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:15:43.722+0000 7f059686b640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T19:15:43.904 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:15:43 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:15:43.730+0000 7f05936a6640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T19:15:44.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[28968]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:44.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[28968]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:44.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[39520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T19:15:44.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[39520]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T19:15:44.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[39520]: from='osd.7 v1:172.21.15.150:6812/3911639935' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:44.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:43 smithi115 ceph-mon[39520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]: dispatch 2024-09-10T19:15:44.842 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:15:44.842 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:44.842 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:44.842 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:15:44.842 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.842 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: osd.7 v1:172.21.15.150:6812/3911639935 boot 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: osd.7 v1:172.21.15.150:6812/3911639935 boot 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:15:44.843 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:44 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T19:15:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi150", "root=default"]}]': finished 2024-09-10T19:15:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T19:15:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: osd.7 v1:172.21.15.150:6812/3911639935 boot 2024-09-10T19:15:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T19:15:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:44 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:15:45.465 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":49,"num_osds":8,"num_up_osds":8,"osd_up_since":1725995744,"num_in_osds":8,"osd_in_since":1725995727,"num_remapped_pgs":1} 2024-09-10T19:15:45.465 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:15:45.729 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:15:45.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: purged_snaps scrub starts 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: purged_snaps scrub ok 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1784534323' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:45.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:45 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: purged_snaps scrub starts 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: purged_snaps scrub ok 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1784534323' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: purged_snaps scrub starts 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: purged_snaps scrub ok 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1784534323' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.972 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:45.973 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:45 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[28968]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[28968]: Detected new or changed devices on smithi150 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[28968]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[28968]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[39520]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:46.982 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[39520]: Detected new or changed devices on smithi150 2024-09-10T19:15:46.983 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T19:15:46.983 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[39520]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:15:46.983 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:46 smithi115 ceph-mon[39520]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:15:47.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:46 smithi150 ceph-mon[33096]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:46 smithi150 ceph-mon[33096]: Detected new or changed devices on smithi150 2024-09-10T19:15:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:46 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 4272M 2024-09-10T19:15:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:46 smithi150 ceph-mon[33096]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:15:47.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:46 smithi150 ceph-mon[33096]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T19:15:47.347 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:15:47.347 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":51,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","created":"2024-09-10T19:10:54.633902+0000","modified":"2024-09-10T19:15:46.294646+0000","last_up_change":"2024-09-10T19:15:44.291743+0000","last_in_change":"2024-09-10T19:15:27.611672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T19:13:49.735959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"004c62f5-ed21-4e2e-aeb6-41ff66d018e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":1864670008}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":1864670008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":1864670008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":1864670008}]},"public_addr":"172.21.15.115:6801/1864670008","cluster_addr":"172.21.15.115:6802/1864670008","heartbeat_back_addr":"172.21.15.115:6804/1864670008","heartbeat_front_addr":"172.21.15.115:6803/1864670008","state":["exists","up"]},{"osd":1,"uuid":"55469ced-c422-4c09-8962-39641a099a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":1865971090}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":1865971090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":1865971090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":1865971090}]},"public_addr":"172.21.15.115:6805/1865971090","cluster_addr":"172.21.15.115:6806/1865971090","heartbeat_back_addr":"172.21.15.115:6808/1865971090","heartbeat_front_addr":"172.21.15.115:6807/1865971090","state":["exists","up"]},{"osd":2,"uuid":"5dc996f6-4d04-4b76-a7a1-333bdf21b1cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":5537849}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":5537849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6812","nonce":5537849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":5537849}]},"public_addr":"172.21.15.115:6809/5537849","cluster_addr":"172.21.15.115:6810/5537849","heartbeat_back_addr":"172.21.15.115:6812/5537849","heartbeat_front_addr":"172.21.15.115:6811/5537849","state":["exists","up"]},{"osd":3,"uuid":"6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6813","nonce":3633830068}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6814","nonce":3633830068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6816","nonce":3633830068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6815","nonce":3633830068}]},"public_addr":"172.21.15.115:6813/3633830068","cluster_addr":"172.21.15.115:6814/3633830068","heartbeat_back_addr":"172.21.15.115:6816/3633830068","heartbeat_front_addr":"172.21.15.115:6815/3633830068","state":["exists","up"]},{"osd":4,"uuid":"0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6800","nonce":2544371837}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6801","nonce":2544371837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6803","nonce":2544371837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6802","nonce":2544371837}]},"public_addr":"172.21.15.150:6800/2544371837","cluster_addr":"172.21.15.150:6801/2544371837","heartbeat_back_addr":"172.21.15.150:6803/2544371837","heartbeat_front_addr":"172.21.15.150:6802/2544371837","state":["exists","up"]},{"osd":5,"uuid":"4265f75c-402b-4094-b27f-dd1854d8dffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6804","nonce":2087992752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6805","nonce":2087992752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6807","nonce":2087992752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6806","nonce":2087992752}]},"public_addr":"172.21.15.150:6804/2087992752","cluster_addr":"172.21.15.150:6805/2087992752","heartbeat_back_addr":"172.21.15.150:6807/2087992752","heartbeat_front_addr":"172.21.15.150:6806/2087992752","state":["exists","up"]},{"osd":6,"uuid":"f7023616-9190-4517-9b6b-63c8112c6f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6808","nonce":1869693771}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6809","nonce":1869693771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6811","nonce":1869693771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6810","nonce":1869693771}]},"public_addr":"172.21.15.150:6808/1869693771","cluster_addr":"172.21.15.150:6809/1869693771","heartbeat_back_addr":"172.21.15.150:6811/1869693771","heartbeat_front_addr":"172.21.15.150:6810/1869693771","state":["exists","up"]},{"osd":7,"uuid":"044ab88d-b207-4094-aa68-ac7908dcea54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6812","nonce":3911639935}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6813","nonce":3911639935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6815","nonce":3911639935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6814","nonce":3911639935}]},"public_addr":"172.21.15.150:6812/3911639935","cluster_addr":"172.21.15.150:6813/3911639935","heartbeat_back_addr":"172.21.15.150:6815/3911639935","heartbeat_front_addr":"172.21.15.150:6814/3911639935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:01.081493+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:23.924903+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:46.056140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:09.375125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:31.152542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:54.863966+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:19.020098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:42.846543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/211420866":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/159701939":"2024-09-11T19:11:45.436250+0000","172.21.15.115:6800/1975523082":"2024-09-11T19:11:45.436250+0000","172.21.15.115:6800/2994657139":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/1293962756":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/156821850":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/2786263605":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/2585189917":"2024-09-11T19:11:23.792323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:15:47.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:47 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1548613571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:15:47.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:47 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1548613571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:15:48.022 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T19:13:49.735959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T19:15:48.023 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T19:15:48.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:47 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1548613571' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:15:48.269 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:15:49.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:48 smithi150 ceph-mon[33096]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:49.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:48 smithi115 ceph-mon[39520]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:48 smithi115 ceph-mon[28968]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:50.029 INFO:teuthology.orchestra.run.smithi115.stdout:pg_num: 1 2024-09-10T19:15:50.748 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi115 2024-09-10T19:15:50.749 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi115=foo.a' 2024-09-10T19:15:51.007 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:51.129 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:50 smithi150 ceph-mon[33096]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:51.129 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:50 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1738885397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:15:51.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:50 smithi115 ceph-mon[39520]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:51.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:50 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1738885397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:15:51.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:50 smithi115 ceph-mon[28968]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:15:51.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:50 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1738885397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T19:15:52.084 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:51 smithi150 ceph-mon[33096]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:15:52.084 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:51 smithi150 ceph-mon[33096]: Cluster is now healthy 2024-09-10T19:15:52.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:51 smithi115 ceph-mon[28968]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:15:52.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:51 smithi115 ceph-mon[28968]: Cluster is now healthy 2024-09-10T19:15:52.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:51 smithi115 ceph-mon[39520]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:15:52.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:51 smithi115 ceph-mon[39520]: Cluster is now healthy 2024-09-10T19:15:52.737 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled rgw.foo.a update... 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:52 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:15:53.144 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:53.145 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:52 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:53.406 DEBUG:teuthology.orchestra.run.smithi115:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T19:15:53.409 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi150 2024-09-10T19:15:53.409 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T19:15:53.661 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:53.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:53 smithi150 ceph-mon[33096]: from='client.24358 v1:172.21.15.150:0/1453970525' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:53.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:53 smithi150 ceph-mon[33096]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:53.957 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:53 smithi150 ceph-mon[33096]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T19:15:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[39520]: from='client.24358 v1:172.21.15.150:0/1453970525' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[39520]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[39520]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T19:15:54.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[28968]: from='client.24358 v1:172.21.15.150:0/1453970525' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi115=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:15:54.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[28968]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:54.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:53 smithi115 ceph-mon[28968]: Deploying daemon rgw.foo.a on smithi115 2024-09-10T19:15:55.025 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:54 smithi150 ceph-mon[33096]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T19:15:55.031 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:54 smithi115 ceph-mon[39520]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T19:15:55.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:54 smithi115 ceph-mon[28968]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T19:15:55.413 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:15:55 smithi115 systemd[1]: Started Ceph rgw.foo.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:15:55.801 INFO:teuthology.orchestra.run.smithi150.stderr:pool 'datapool' created 2024-09-10T19:15:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/1725798543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.136 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.137 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:55 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/1725798543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/1725798543' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:56.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:55 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:15:56.450 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T19:15:56.695 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:15:57.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:56 smithi150 ceph-mon[33096]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:15:57.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:56 smithi150 ceph-mon[33096]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:57.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:56 smithi150 ceph-mon[33096]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:15:57.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:56 smithi150 ceph-mon[33096]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:15:57.064 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:56 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:15:57.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[28968]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[28968]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[28968]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[28968]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[39520]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[39520]: Saving service rgw.foo.a spec with placement smithi115=foo.a;count:1 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[39520]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T19:15:57.211 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[39520]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T19:15:57.212 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:56 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T19:15:57.991 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:15:57.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:15:57.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:57.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:57.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:57.992 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:57 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:58.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:15:58.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:15:58.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:58.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:58.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:15:58.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:57 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: Checking dashboard <-> RGW credentials 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: pgmap v169: 36 pgs: 16 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/2339759521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:15:59.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:15:58 smithi150 ceph-mon[33096]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:15:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: Checking dashboard <-> RGW credentials 2024-09-10T19:15:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: pgmap v169: 36 pgs: 16 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:15:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:15:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/2339759521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[39520]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: Checking dashboard <-> RGW credentials 2024-09-10T19:15:59.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: pgmap v169: 36 pgs: 16 creating+peering, 19 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/2339759521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T19:15:59.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:15:58 smithi115 ceph-mon[28968]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T19:16:01.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: pgmap v172: 68 pgs: 3 creating+activating, 25 creating+peering, 23 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T19:16:01.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: Cluster is now healthy 2024-09-10T19:16:01.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:00 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: pgmap v172: 68 pgs: 3 creating+activating, 25 creating+peering, 23 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: Cluster is now healthy 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: pgmap v172: 68 pgs: 3 creating+activating, 25 creating+peering, 23 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/290670629' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: Cluster is now healthy 2024-09-10T19:16:01.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:00 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:01.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.150 --placement '1;smithi150=iscsi.a' 2024-09-10T19:16:01.697 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:01 smithi150 ceph-mon[33096]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:01 smithi150 ceph-mon[33096]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:16:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[28968]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[28968]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:16:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1381367830' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[39520]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T19:16:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:01 smithi115 ceph-mon[39520]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T19:16:02.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: pgmap v175: 100 pgs: 3 creating+activating, 16 creating+peering, 32 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T19:16:02.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:16:02.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:02.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:02.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:02.885 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:02 smithi150 ceph-mon[33096]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: pgmap v175: 100 pgs: 3 creating+activating, 16 creating+peering, 32 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T19:16:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[28968]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: pgmap v175: 100 pgs: 3 creating+activating, 16 creating+peering, 32 unknown, 49 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:02 smithi115 ceph-mon[39520]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T19:16:03.283 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled iscsi.datapool update... 2024-09-10T19:16:03.882 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T19:16:03.883 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:16:03.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T19:16:03.926 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:16:03.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T19:16:03.958 DEBUG:teuthology.orchestra.run.smithi150:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T19:16:04.001 INFO:tasks.cephadm:Adding prometheus.a on smithi150 2024-09-10T19:16:04.001 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi150=a' 2024-09-10T19:16:04.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:03 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:16:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1662237145' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2898857295' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.34406 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='client.24409 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T19:16:04.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:03 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:04.315 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:05.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='client.24421 v1:172.21.15.150:0/1313926140' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:05.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T19:16:05.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: pgmap v178: 132 pgs: 4 creating+activating, 4 creating+peering, 35 unknown, 89 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T19:16:05.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.087 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:05.088 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:04 smithi150 ceph-mon[33096]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T19:16:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='client.24421 v1:172.21.15.150:0/1313926140' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: pgmap v178: 132 pgs: 4 creating+activating, 4 creating+peering, 35 unknown, 89 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[28968]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T19:16:05.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='client.24421 v1:172.21.15.150:0/1313926140' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.150", "placement": "1;smithi150=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: Saving service iscsi.datapool spec with placement smithi150=iscsi.a;count:1 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: pgmap v178: 132 pgs: 4 creating+activating, 4 creating+peering, 35 unknown, 89 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='client.34406 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='client.24409 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:04 smithi115 ceph-mon[39520]: Deploying daemon iscsi.iscsi.a on smithi150 2024-09-10T19:16:07.105 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:06 smithi150 ceph-mon[33096]: pgmap v180: 132 pgs: 1 creating+activating, 4 creating+peering, 10 unknown, 117 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 2.4 KiB/s wr, 40 op/s 2024-09-10T19:16:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:06 smithi115 ceph-mon[28968]: pgmap v180: 132 pgs: 1 creating+activating, 4 creating+peering, 10 unknown, 117 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 2.4 KiB/s wr, 40 op/s 2024-09-10T19:16:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:06 smithi115 ceph-mon[39520]: pgmap v180: 132 pgs: 1 creating+activating, 4 creating+peering, 10 unknown, 117 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 2.4 KiB/s wr, 40 op/s 2024-09-10T19:16:07.279 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled prometheus update... 2024-09-10T19:16:07.358 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:07 smithi150 podman[59710]: 2024-09-10 19:16:07.229899914 +0000 UTC m=+0.020235822 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:16:07.632 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:07 smithi150 podman[59710]: 2024-09-10 19:16:07.357560379 +0000 UTC m=+0.147896277 container create b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T19:16:08.037 DEBUG:teuthology.orchestra.run.smithi150:prometheus.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T19:16:08.040 INFO:tasks.cephadm:Adding node-exporter.a on smithi115 2024-09-10T19:16:08.040 INFO:tasks.cephadm:Adding node-exporter.b on smithi150 2024-09-10T19:16:08.040 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi115=a;smithi150=b' 2024-09-10T19:16:08.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:08 smithi150 podman[59710]: 2024-09-10 19:16:08.149242895 +0000 UTC m=+0.939578795 container init b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T19:16:08.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:08 smithi150 podman[59710]: 2024-09-10 19:16:08.15274826 +0000 UTC m=+0.943084165 container start b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T19:16:08.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[59760]: log file path now is '/var/log/tcmu-runner.log' 2024-09-10T19:16:08.562 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:08.632 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:08 smithi150 bash[59710]: b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 2024-09-10T19:16:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:08 smithi150 ceph-mon[33096]: from='client.24442 v1:172.21.15.150:0/2021420226' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:08 smithi150 ceph-mon[33096]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T19:16:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:08 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:08 smithi150 ceph-mon[33096]: pgmap v181: 132 pgs: 1 creating+activating, 4 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.2 KiB/s wr, 119 op/s 2024-09-10T19:16:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[28968]: from='client.24442 v1:172.21.15.150:0/2021420226' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[28968]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T19:16:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[28968]: pgmap v181: 132 pgs: 1 creating+activating, 4 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.2 KiB/s wr, 119 op/s 2024-09-10T19:16:08.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[39520]: from='client.24442 v1:172.21.15.150:0/2021420226' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:08.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[39520]: Saving service prometheus spec with placement smithi150=a;count:1 2024-09-10T19:16:08.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:08.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:08 smithi115 ceph-mon[39520]: pgmap v181: 132 pgs: 1 creating+activating, 4 creating+peering, 127 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.2 KiB/s wr, 119 op/s 2024-09-10T19:16:08.951 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:08 smithi150 podman[59898]: 2024-09-10 19:16:08.836897747 +0000 UTC m=+0.018604179 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:16:09.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 podman[59898]: 2024-09-10 19:16:09.352483869 +0000 UTC m=+0.534190301 container create e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:16:09.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 podman[59898]: 2024-09-10 19:16:09.563161716 +0000 UTC m=+0.744868148 container init e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T19:16:09.633 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 podman[59898]: 2024-09-10 19:16:09.566224417 +0000 UTC m=+0.747930849 container start e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T19:16:10.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 bash[59898]: e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a 2024-09-10T19:16:10.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 systemd[1]: Started Ceph iscsi.iscsi.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:16:10.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Started the configuration object watcher 2024-09-10T19:16:10.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Checking for config object changes every 1s 2024-09-10T19:16:10.125 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Processing osd blocklist entries for this node 2024-09-10T19:16:10.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:10.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/3939866669' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:16:10.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Reading the configuration object to update local LIO configuration 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Configuration does not have an entry for this host(smithi150.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: * Environment: production 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: Use a production WSGI server instead. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: * Debug mode: off 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug * Running on all addresses. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: * Running on all addresses. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug * Running on http://[fe80::ec4:7aff:fe88:6e35]:5000/ (Press CTRL+C to quit) 2024-09-10T19:16:10.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: * Running on http://[fe80::ec4:7aff:fe88:6e35]:5000/ (Press CTRL+C to quit) 2024-09-10T19:16:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:16:10.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/3939866669' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:10.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:10 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/3939866669' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T19:16:10.824 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled node-exporter update... 2024-09-10T19:16:11.777 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:11 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:11.778 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:11 smithi150 ceph-mon[33096]: from='client.24478 v1:172.21.15.150:0/3843976847' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:11.778 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:11 smithi150 ceph-mon[33096]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T19:16:11.778 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:11 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:11.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:11.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[28968]: from='client.24478 v1:172.21.15.150:0/3843976847' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:11.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[28968]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T19:16:11.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:11.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:11.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[39520]: from='client.24478 v1:172.21.15.150:0/3843976847' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi115=a;smithi150=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:11.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[39520]: Saving service node-exporter spec with placement smithi115=a;smithi150=b;count:2 2024-09-10T19:16:11.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:11 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:12.112 DEBUG:teuthology.orchestra.run.smithi115:node-exporter.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T19:16:12.115 DEBUG:teuthology.orchestra.run.smithi150:node-exporter.b> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T19:16:12.117 INFO:tasks.cephadm:Adding alertmanager.a on smithi115 2024-09-10T19:16:12.118 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi115=a' 2024-09-10T19:16:13.046 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:12 smithi150 ceph-mon[33096]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-10T19:16:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:12 smithi150 ceph-mon[33096]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:16:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:12 smithi115 ceph-mon[28968]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-10T19:16:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:12 smithi115 ceph-mon[28968]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:16:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:12 smithi115 ceph-mon[39520]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-10T19:16:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:12 smithi115 ceph-mon[39520]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T19:16:14.232 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:14.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:14.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.233 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:13 smithi150 ceph-mon[33096]: Deploying daemon prometheus.a on smithi150 2024-09-10T19:16:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[28968]: Deploying daemon prometheus.a on smithi150 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:14.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:13 smithi115 ceph-mon[39520]: Deploying daemon prometheus.a on smithi150 2024-09-10T19:16:14.778 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled alertmanager update... 2024-09-10T19:16:15.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:14 smithi150 ceph-mon[33096]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 116 op/s 2024-09-10T19:16:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:14 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:15.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:14 smithi115 ceph-mon[28968]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 116 op/s 2024-09-10T19:16:15.315 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:14 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:15.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:14 smithi115 ceph-mon[39520]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.6 KiB/s wr, 116 op/s 2024-09-10T19:16:15.315 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:14 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:15.547 DEBUG:teuthology.orchestra.run.smithi115:alertmanager.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T19:16:15.550 INFO:tasks.cephadm:Adding grafana.a on smithi150 2024-09-10T19:16:15.550 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi150=a' 2024-09-10T19:16:15.802 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:16.278 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:15 smithi150 ceph-mon[33096]: from='client.24484 v1:172.21.15.150:0/2651617083' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:15 smithi150 ceph-mon[33096]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T19:16:16.279 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:15 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[28968]: from='client.24484 v1:172.21.15.150:0/2651617083' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[28968]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T19:16:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:16.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[39520]: from='client.24484 v1:172.21.15.150:0/2651617083' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi115=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:16.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[39520]: Saving service alertmanager spec with placement smithi115=a;count:1 2024-09-10T19:16:16.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:15 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:17.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:16 smithi115 ceph-mon[28968]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 2.8 KiB/s wr, 95 op/s 2024-09-10T19:16:17.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:16 smithi115 ceph-mon[39520]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 2.8 KiB/s wr, 95 op/s 2024-09-10T19:16:17.243 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:16 smithi150 ceph-mon[33096]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 39 KiB/s rd, 2.8 KiB/s wr, 95 op/s 2024-09-10T19:16:18.078 INFO:teuthology.orchestra.run.smithi150.stdout:Scheduled grafana update... 2024-09-10T19:16:19.082 DEBUG:teuthology.orchestra.run.smithi150:grafana.a> sudo journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T19:16:19.085 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T19:16:19.085 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T19:16:19.334 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:19 smithi150 ceph-mon[33096]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 41 op/s 2024-09-10T19:16:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:19 smithi150 ceph-mon[33096]: from='client.24490 v1:172.21.15.150:0/967985007' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:19 smithi150 ceph-mon[33096]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T19:16:19.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:19 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:19.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[28968]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 41 op/s 2024-09-10T19:16:19.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[28968]: from='client.24490 v1:172.21.15.150:0/967985007' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:19.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[28968]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T19:16:19.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:19.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[39520]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 41 op/s 2024-09-10T19:16:19.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[39520]: from='client.24490 v1:172.21.15.150:0/967985007' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi150=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:16:19.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[39520]: Saving service grafana spec with placement smithi150=a;count:1 2024-09-10T19:16:19.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:19 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:20.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:16:21.190 INFO:teuthology.orchestra.run.smithi115.stdout:[client.0] 2024-09-10T19:16:21.190 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQAFm+Bm0ZUPCxAA5yG3+Sokq5EMZcDVR12n0w== 2024-09-10T19:16:21.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:21 smithi150 ceph-mon[33096]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 703 B/s rd, 100 B/s wr, 1 op/s 2024-09-10T19:16:21.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:21 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:21 smithi115 ceph-mon[28968]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 703 B/s rd, 100 B/s wr, 1 op/s 2024-09-10T19:16:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:21 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:21.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:21 smithi115 ceph-mon[39520]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 703 B/s rd, 100 B/s wr, 1 op/s 2024-09-10T19:16:21.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:21 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:21.832 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2024-09-10T19:16:21.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T19:16:21.832 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T19:16:21.870 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T19:16:22.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:22 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:22.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:22 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:22.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:22 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:22.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:22 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:22.326 INFO:teuthology.orchestra.run.smithi150.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T19:16:22.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:22 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:22.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:22 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1835057926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:23.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:23 smithi150 ceph-mon[33096]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:23 smithi115 ceph-mon[28968]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:23.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:23 smithi115 ceph-mon[39520]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 597 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:25 smithi150 ceph-mon[33096]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:25.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:25 smithi115 ceph-mon[28968]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:25.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:25 smithi115 ceph-mon[39520]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T19:16:26.939 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2024-09-10T19:16:26.939 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQAKm+BmQhSaNxAAlgwpx3vMXpvuu64lVPB6ow== 2024-09-10T19:16:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-mon[33096]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:16:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.150:0/146747920' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-mon[33096]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-mon[33096]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[28968]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:16:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.150:0/146747920' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[28968]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[28968]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:27.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[39520]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:16:27.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.150:0/146747920' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[39520]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T19:16:27.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:27 smithi115 ceph-mon[39520]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T19:16:27.764 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2024-09-10T19:16:27.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T19:16:27.765 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T19:16:27.804 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T19:16:27.805 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T19:16:27.805 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T19:16:27.998 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:16:27.982Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T19:16:27.999 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:16:27.997Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.499783ms db_storage=1.132µs remote_storage=1.636µs web_handler=478ns query_engine=660ns scrape=405.893µs scrape_sd=33.067µs notify=998ns notify_sd=2.826µs rules=14.703279ms tracing=7.685µs 2024-09-10T19:16:27.999 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:16:27.997Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T19:16:27.999 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:27 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:16:27.997Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T19:16:28.055 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:28.382 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:28 smithi150 bash[61438]: e8c9e71ecca18b26ff58ecfda8ae892f6800243f951e5c2b6783c7967c777a6a 2024-09-10T19:16:28.382 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:16:28 smithi150 systemd[1]: Started Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:16:29.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ignoring --setuser ceph since I am not root 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ignoring --setgroup ceph since I am not root 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.261+0000 7f7d950a9ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.317+0000 7f7d950a9ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-mon[33096]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:16:29.387 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[28968]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[39520]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' 2024-09-10T19:16:29.388 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T19:16:29.389 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setuser ceph since I am not root 2024-09-10T19:16:29.389 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setgroup ceph since I am not root 2024-09-10T19:16:29.389 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.265+0000 7f764f652ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:16:29.389 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.322+0000 7f764f652ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:16:29.389 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.386+0000 7f764f652ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:16:29.645 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.382+0000 7f7d950a9ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:16:29.645 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.506+0000 7f7d950a9ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:16:29.648 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.519+0000 7f764f652ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:16:29.877 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:16:29.900 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.643+0000 7f7d950a9ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:16:29.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.670+0000 7f764f652ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:16:30.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[28968]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:16:30.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[28968]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:16:30.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2692024528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:16:30.165 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:29.941+0000 7f764f652ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:16:30.166 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:30.001+0000 7f764f652ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:16:30.166 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[39520]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:16:30.166 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[39520]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:16:30.166 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2692024528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:16:30.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:30 smithi150 ceph-mon[33096]: from='mgr.14150 v1:172.21.15.115:0/2427341884' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T19:16:30.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:30 smithi150 ceph-mon[33096]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T19:16:30.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:30 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2692024528' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T19:16:30.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.898+0000 7f7d950a9ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:16:30.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:29 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:29.957+0000 7f7d950a9ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:16:30.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:30 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:30.119+0000 7f7d950a9ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:16:30.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:30 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:30.163+0000 7f764f652ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:16:30.725 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6800","nonce":1677419396}]},"active_addr":"172.21.15.115:6800/1677419396","active_change":"2024-09-10T19:11:45.436380+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.115:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3767908429}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":2808659357}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":29395105}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":1792956449}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.115:0","nonce":3757843238}]}]} 2024-09-10T19:16:30.732 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T19:16:30.732 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T19:16:30.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:16:30.990 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:31.349 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:31.049+0000 7f764f652ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:16:31.349 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:31.120+0000 7f764f652ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:16:31.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:30 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:30.969+0000 7f7d950a9ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:16:31.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:31 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:31.034+0000 7f7d950a9ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:16:32.036 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:31 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:31.746+0000 7f7d950a9ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:16:32.036 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:31 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:31.856+0000 7f7d950a9ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:16:32.036 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:31 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:31.974+0000 7f7d950a9ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:16:32.037 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.034+0000 7f7d950a9ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:16:32.152 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:31.848+0000 7f764f652ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:16:32.152 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:31.960+0000 7f764f652ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:16:32.152 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.082+0000 7f764f652ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:16:32.152 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.148+0000 7f764f652ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:16:32.287 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.161+0000 7f7d950a9ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:16:32.287 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.228+0000 7f7d950a9ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:16:32.411 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.277+0000 7f764f652ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:16:32.412 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.349+0000 7f764f652ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:16:32.412 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.410+0000 7f764f652ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:16:32.548 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.285+0000 7f7d950a9ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:16:32.549 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.420+0000 7f7d950a9ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:16:32.549 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.483+0000 7f7d950a9ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:16:32.549 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.547+0000 7f7d950a9ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:16:32.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.547+0000 7f764f652ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:16:32.663 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.640+0000 7f764f652ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:16:32.768 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:16:32.769 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":61,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","created":"2024-09-10T19:10:54.633902+0000","modified":"2024-09-10T19:16:09.305309+0000","last_up_change":"2024-09-10T19:15:44.291743+0000","last_in_change":"2024-09-10T19:15:27.611672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T19:13:49.735959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T19:15:55.394365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T19:15:55.630990+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T19:15:57.096657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T19:15:59.069885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T19:16:01.131158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"004c62f5-ed21-4e2e-aeb6-41ff66d018e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":1864670008}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":1864670008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":1864670008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":1864670008}]},"public_addr":"172.21.15.115:6801/1864670008","cluster_addr":"172.21.15.115:6802/1864670008","heartbeat_back_addr":"172.21.15.115:6804/1864670008","heartbeat_front_addr":"172.21.15.115:6803/1864670008","state":["exists","up"]},{"osd":1,"uuid":"55469ced-c422-4c09-8962-39641a099a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":1865971090}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":1865971090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":1865971090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":1865971090}]},"public_addr":"172.21.15.115:6805/1865971090","cluster_addr":"172.21.15.115:6806/1865971090","heartbeat_back_addr":"172.21.15.115:6808/1865971090","heartbeat_front_addr":"172.21.15.115:6807/1865971090","state":["exists","up"]},{"osd":2,"uuid":"5dc996f6-4d04-4b76-a7a1-333bdf21b1cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":5537849}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":5537849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6812","nonce":5537849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":5537849}]},"public_addr":"172.21.15.115:6809/5537849","cluster_addr":"172.21.15.115:6810/5537849","heartbeat_back_addr":"172.21.15.115:6812/5537849","heartbeat_front_addr":"172.21.15.115:6811/5537849","state":["exists","up"]},{"osd":3,"uuid":"6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6813","nonce":3633830068}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6814","nonce":3633830068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6816","nonce":3633830068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6815","nonce":3633830068}]},"public_addr":"172.21.15.115:6813/3633830068","cluster_addr":"172.21.15.115:6814/3633830068","heartbeat_back_addr":"172.21.15.115:6816/3633830068","heartbeat_front_addr":"172.21.15.115:6815/3633830068","state":["exists","up"]},{"osd":4,"uuid":"0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6800","nonce":2544371837}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6801","nonce":2544371837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6803","nonce":2544371837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6802","nonce":2544371837}]},"public_addr":"172.21.15.150:6800/2544371837","cluster_addr":"172.21.15.150:6801/2544371837","heartbeat_back_addr":"172.21.15.150:6803/2544371837","heartbeat_front_addr":"172.21.15.150:6802/2544371837","state":["exists","up"]},{"osd":5,"uuid":"4265f75c-402b-4094-b27f-dd1854d8dffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6804","nonce":2087992752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6805","nonce":2087992752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6807","nonce":2087992752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6806","nonce":2087992752}]},"public_addr":"172.21.15.150:6804/2087992752","cluster_addr":"172.21.15.150:6805/2087992752","heartbeat_back_addr":"172.21.15.150:6807/2087992752","heartbeat_front_addr":"172.21.15.150:6806/2087992752","state":["exists","up"]},{"osd":6,"uuid":"f7023616-9190-4517-9b6b-63c8112c6f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6808","nonce":1869693771}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6809","nonce":1869693771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6811","nonce":1869693771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6810","nonce":1869693771}]},"public_addr":"172.21.15.150:6808/1869693771","cluster_addr":"172.21.15.150:6809/1869693771","heartbeat_back_addr":"172.21.15.150:6811/1869693771","heartbeat_front_addr":"172.21.15.150:6810/1869693771","state":["exists","up"]},{"osd":7,"uuid":"044ab88d-b207-4094-aa68-ac7908dcea54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6812","nonce":3911639935}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6813","nonce":3911639935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6815","nonce":3911639935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6814","nonce":3911639935}]},"public_addr":"172.21.15.150:6812/3911639935","cluster_addr":"172.21.15.150:6813/3911639935","heartbeat_back_addr":"172.21.15.150:6815/3911639935","heartbeat_front_addr":"172.21.15.150:6814/3911639935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:01.081493+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:23.924903+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:46.056140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:09.375125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:31.152542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:54.863966+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:19.020098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:42.846543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/211420866":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/159701939":"2024-09-11T19:11:45.436250+0000","172.21.15.115:6800/1975523082":"2024-09-11T19:11:45.436250+0000","172.21.15.115:6800/2994657139":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/1293962756":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/156821850":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/2786263605":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/2585189917":"2024-09-11T19:11:23.792323+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:16:32.820 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.628+0000 7f7d950a9ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:16:32.820 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.685+0000 7f7d950a9ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:16:32.820 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:32.741+0000 7f7d950a9ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:16:33.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:32 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/442792593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:33.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/442792593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:33.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/442792593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:33.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.706+0000 7f764f652ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:16:33.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.797+0000 7f764f652ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:16:33.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.859+0000 7f764f652ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:16:33.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:32 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:32.916+0000 7f764f652ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:16:33.512 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T19:16:33.512 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T19:16:33.757 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:33.676+0000 7f764f652ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:16:33.776 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:33.491+0000 7f7d950a9ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:16:33] ENGINE Bus STARTING 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: CherryPy Checker: 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: The Application mounted at '' has an empty config. 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:16:33] ENGINE Serving on http://:::9283 2024-09-10T19:16:33.822 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:16:33] ENGINE Bus STARTED 2024-09-10T19:16:34.024 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:16:33] ENGINE Bus STARTING 2024-09-10T19:16:34.024 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: CherryPy Checker: 2024-09-10T19:16:34.024 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: The Application mounted at '' has an empty config. 2024-09-10T19:16:34.024 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:34.024 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:16:33] ENGINE Serving on http://:::9283 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:16:33] ENGINE Bus STARTED 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: Standby manager daemon x restarted 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: Standby manager daemon x started 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:16:34.025 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: Active manager daemon y restarted 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: mgrmap e18: y(active, starting, since 0.0121791s), standbys: x 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:16:34.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: Manager daemon y is now available 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:34.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: Standby manager daemon x restarted 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: Standby manager daemon x started 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:16:34.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: Active manager daemon y restarted 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: Activating manager daemon y 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: mgrmap e18: y(active, starting, since 0.0121791s), standbys: x 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:16:34.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: Manager daemon y is now available 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.030 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:34.031 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: Standby manager daemon x restarted 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: Standby manager daemon x started 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: Active manager daemon y restarted 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: Activating manager daemon y 2024-09-10T19:16:34.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: mgrmap e18: y(active, starting, since 0.0121791s), standbys: x 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:16:34.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: Manager daemon y is now available 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:34.135 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:16:35.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:35.153+0000 7f75f2414640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:16:35] ENGINE Error in HTTPServer.serve 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Traceback (most recent call last): 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._connections.run(self.expiration_interval) 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._run(expiration_interval) 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:16:35.414 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: s = self.context.wrap_socket( 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: return self.sslsocket_class._create( 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self.do_handshake() 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: self._sslobj.do_handshake() 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:16:35.415 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:16:35.580 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:16:35.580 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":62,"fsid":"4f14fa52-6fa8-11ef-bcea-c7b262605968","created":"2024-09-10T19:10:54.633902+0000","modified":"2024-09-10T19:16:33.679203+0000","last_up_change":"2024-09-10T19:15:44.291743+0000","last_in_change":"2024-09-10T19:15:27.611672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T19:13:49.735959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T19:15:55.394365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T19:15:55.630990+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":57,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T19:15:57.096657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T19:15:59.069885+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T19:16:01.131158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"004c62f5-ed21-4e2e-aeb6-41ff66d018e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6801","nonce":1864670008}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6802","nonce":1864670008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6804","nonce":1864670008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6803","nonce":1864670008}]},"public_addr":"172.21.15.115:6801/1864670008","cluster_addr":"172.21.15.115:6802/1864670008","heartbeat_back_addr":"172.21.15.115:6804/1864670008","heartbeat_front_addr":"172.21.15.115:6803/1864670008","state":["exists","up"]},{"osd":1,"uuid":"55469ced-c422-4c09-8962-39641a099a5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6805","nonce":1865971090}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6806","nonce":1865971090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6808","nonce":1865971090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6807","nonce":1865971090}]},"public_addr":"172.21.15.115:6805/1865971090","cluster_addr":"172.21.15.115:6806/1865971090","heartbeat_back_addr":"172.21.15.115:6808/1865971090","heartbeat_front_addr":"172.21.15.115:6807/1865971090","state":["exists","up"]},{"osd":2,"uuid":"5dc996f6-4d04-4b76-a7a1-333bdf21b1cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6809","nonce":5537849}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6810","nonce":5537849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6812","nonce":5537849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6811","nonce":5537849}]},"public_addr":"172.21.15.115:6809/5537849","cluster_addr":"172.21.15.115:6810/5537849","heartbeat_back_addr":"172.21.15.115:6812/5537849","heartbeat_front_addr":"172.21.15.115:6811/5537849","state":["exists","up"]},{"osd":3,"uuid":"6ffa3efc-0e38-42e0-875d-c2f0c0d7d2b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6813","nonce":3633830068}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6814","nonce":3633830068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6816","nonce":3633830068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.115:6815","nonce":3633830068}]},"public_addr":"172.21.15.115:6813/3633830068","cluster_addr":"172.21.15.115:6814/3633830068","heartbeat_back_addr":"172.21.15.115:6816/3633830068","heartbeat_front_addr":"172.21.15.115:6815/3633830068","state":["exists","up"]},{"osd":4,"uuid":"0bb077b2-bbc5-4dc9-82d2-0a265e4fb20c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6800","nonce":2544371837}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6801","nonce":2544371837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6803","nonce":2544371837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6802","nonce":2544371837}]},"public_addr":"172.21.15.150:6800/2544371837","cluster_addr":"172.21.15.150:6801/2544371837","heartbeat_back_addr":"172.21.15.150:6803/2544371837","heartbeat_front_addr":"172.21.15.150:6802/2544371837","state":["exists","up"]},{"osd":5,"uuid":"4265f75c-402b-4094-b27f-dd1854d8dffe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6804","nonce":2087992752}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6805","nonce":2087992752}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6807","nonce":2087992752}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6806","nonce":2087992752}]},"public_addr":"172.21.15.150:6804/2087992752","cluster_addr":"172.21.15.150:6805/2087992752","heartbeat_back_addr":"172.21.15.150:6807/2087992752","heartbeat_front_addr":"172.21.15.150:6806/2087992752","state":["exists","up"]},{"osd":6,"uuid":"f7023616-9190-4517-9b6b-63c8112c6f92","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6808","nonce":1869693771}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6809","nonce":1869693771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6811","nonce":1869693771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6810","nonce":1869693771}]},"public_addr":"172.21.15.150:6808/1869693771","cluster_addr":"172.21.15.150:6809/1869693771","heartbeat_back_addr":"172.21.15.150:6811/1869693771","heartbeat_front_addr":"172.21.15.150:6810/1869693771","state":["exists","up"]},{"osd":7,"uuid":"044ab88d-b207-4094-aa68-ac7908dcea54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6812","nonce":3911639935}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6813","nonce":3911639935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6815","nonce":3911639935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.150:6814","nonce":3911639935}]},"public_addr":"172.21.15.150:6812/3911639935","cluster_addr":"172.21.15.150:6813/3911639935","heartbeat_back_addr":"172.21.15.150:6815/3911639935","heartbeat_front_addr":"172.21.15.150:6814/3911639935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:01.081493+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:23.924903+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:13:46.056140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:09.375125+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:31.152542+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:14:54.863966+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:19.020098+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T19:15:42.846543+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.115:0/2585189917":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/3757843238":"2024-09-11T19:16:33.679157+0000","172.21.15.115:0/2786263605":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/1293962756":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/156821850":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/1792956449":"2024-09-11T19:16:33.679157+0000","172.21.15.115:6800/1975523082":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/159701939":"2024-09-11T19:11:45.436250+0000","172.21.15.115:6800/1677419396":"2024-09-11T19:16:33.679157+0000","172.21.15.115:6800/2994657139":"2024-09-11T19:11:23.792323+0000","172.21.15.115:0/211420866":"2024-09-11T19:11:45.436250+0000","172.21.15.115:0/3767908429":"2024-09-11T19:16:33.679157+0000","172.21.15.115:0/29395105":"2024-09-11T19:16:33.679157+0000","172.21.15.115:0/2808659357":"2024-09-11T19:16:33.679157+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T19:16:35.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: mgrmap e19: y(active, since 1.01361s), standbys: x 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: [10/Sep/2024:19:16:35] ENGINE Bus STARTING 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: [10/Sep/2024:19:16:35] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: [10/Sep/2024:19:16:35] ENGINE Bus STARTED 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: [10/Sep/2024:19:16:35] ENGINE Error in HTTPServer.serve 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: Traceback (most recent call last): 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._run(expiration_interval) 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:16:35.883 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: s = self.context.wrap_socket( 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: return self.sslsocket_class._create( 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: self.do_handshake() 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._sslobj.do_handshake() 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:35.884 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:35 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2637963422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:36.018 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: mgrmap e19: y(active, since 1.01361s), standbys: x 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: [10/Sep/2024:19:16:35] ENGINE Bus STARTING 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: [10/Sep/2024:19:16:35] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: [10/Sep/2024:19:16:35] ENGINE Bus STARTED 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: [10/Sep/2024:19:16:35] ENGINE Error in HTTPServer.serve 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:16:36.019 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2637963422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:36.020 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: mgrmap e19: y(active, since 1.01361s), standbys: x 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: [10/Sep/2024:19:16:35] ENGINE Bus STARTING 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: [10/Sep/2024:19:16:35] ENGINE Serving on https://172.21.15.115:7150 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: [10/Sep/2024:19:16:35] ENGINE Bus STARTED 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: [10/Sep/2024:19:16:35] ENGINE Error in HTTPServer.serve 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:16:36.021 INFO:journalctl@ceph.mon.c.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: self.do_handshake() 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:36.022 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:35 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2637963422' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T19:16:36.664 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T19:16:36.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T19:16:36.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T19:16:36.665 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T19:16:36.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T19:16:36.666 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T19:16:36.667 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T19:16:36.667 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T19:16:37.238 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.251 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.256 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.258 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.258 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.261 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.270 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.293 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[28968]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[28968]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T19:16:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:37.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:37.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[39520]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:37.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[39520]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T19:16:37.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:37.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:37 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:37 smithi150 ceph-mon[33096]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:37 smithi150 ceph-mon[33096]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T19:16:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:37 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:37.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:37 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T19:16:38.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[28968]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:38.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:38 smithi115 ceph-mon[39520]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:38.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 3760M 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:38.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-mon[33096]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:16:39.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:16:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:16:39.607 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:39 smithi115 ceph-mon[39520]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T19:16:39.607 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:39 smithi115 ceph-mon[28968]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T19:16:39.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:39 smithi150 ceph-mon[33096]: mgrmap e21: y(active, since 4s), standbys: x 2024-09-10T19:16:41.244 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[39520]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:16:41.244 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:41.244 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:41.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[28968]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:16:41.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:41.245 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:40 smithi150 ceph-mon[33096]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:16:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:41.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:43.233 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:42 smithi115 ceph-mon[28968]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T19:16:43.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:42 smithi115 ceph-mon[39520]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T19:16:43.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:42 smithi150 ceph-mon[33096]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T19:16:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.385 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 3760M 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:44.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:44 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.077 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.100 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[28968]: Deploying daemon alertmanager.a on smithi115 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.101 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:45 smithi115 ceph-mon[39520]: Deploying daemon alertmanager.a on smithi115 2024-09-10T19:16:45.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:16:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:45 smithi150 ceph-mon[33096]: Deploying daemon alertmanager.a on smithi115 2024-09-10T19:16:46.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:46 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:46 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:46.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:46 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:47.343 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:47 smithi115 ceph-mon[28968]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:47.344 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:47 smithi115 ceph-mon[39520]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:47.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:47 smithi150 ceph-mon[33096]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:47.539 INFO:teuthology.orchestra.run.smithi115.stdout:34359738415 2024-09-10T19:16:47.540 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T19:16:48.345 INFO:teuthology.orchestra.run.smithi115.stdout:133143986205 2024-09-10T19:16:48.346 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T19:16:49.087 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:16:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:16:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:49 smithi115 ceph-mon[28968]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:16:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:49 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:49.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:49 smithi115 ceph-mon[39520]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:16:49.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:49 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:49 smithi150 ceph-mon[33096]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T19:16:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:49 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:16:51.348 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:51 smithi115 ceph-mon[28968]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:51.348 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:51 smithi115 ceph-mon[39520]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:51 smithi150 ceph-mon[33096]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T19:16:53.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:53 smithi150 ceph-mon[33096]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:53.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:53 smithi115 ceph-mon[39520]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:53.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:53 smithi115 ceph-mon[28968]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:53.449 INFO:teuthology.orchestra.run.smithi115.stdout:55834574891 2024-09-10T19:16:53.450 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T19:16:55.094 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:16:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:16:55.362 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:55 smithi150 ceph-mon[33096]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:55.389 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:55 smithi115 ceph-mon[28968]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:55.389 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:55 smithi115 ceph-mon[39520]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:16:55.411 INFO:teuthology.orchestra.run.smithi115.stdout:210453397519 2024-09-10T19:16:55.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T19:16:56.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:56 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:56.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:56 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:56.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:56 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:16:56.644 INFO:teuthology.orchestra.run.smithi115.stdout:158913789978 2024-09-10T19:16:56.645 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T19:16:57.116 INFO:teuthology.orchestra.run.smithi115.stdout:111669149730 2024-09-10T19:16:57.116 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T19:16:57.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:57 smithi150 ceph-mon[33096]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:57.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:57 smithi115 ceph-mon[28968]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:57.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:57 smithi115 ceph-mon[39520]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:16:57.882 INFO:teuthology.orchestra.run.smithi115.stdout:77309411369 2024-09-10T19:16:57.883 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T19:16:58.721 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:59.085 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:16:59.099 INFO:teuthology.orchestra.run.smithi115.stdout:184683593748 2024-09-10T19:16:59.100 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T19:16:59.102 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:16:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:16:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:16:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:16:59 smithi115 ceph-mon[28968]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:16:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:16:59 smithi115 ceph-mon[39520]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:16:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:16:59 smithi150 ceph-mon[33096]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:01.192 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:01 smithi115 ceph-mon[39520]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:01.192 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:01 smithi115 ceph-mon[28968]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:01.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:01 smithi150 ceph-mon[33096]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:01.723 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:01.723 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:01.730 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:02.942 INFO:teuthology.orchestra.run.smithi115.stdout:34359738418 2024-09-10T19:17:03.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:03 smithi115 ceph-mon[28968]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:03.236 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:03 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2001018283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:17:03.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:03 smithi115 ceph-mon[39520]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:03.236 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:03 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2001018283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:17:03.331 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:03 smithi150 ceph-mon[33096]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:03 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2001018283' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T19:17:03.716 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:03.716 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:04.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:04 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:04.384 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:04 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:04.385 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:04 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:05.113 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:05.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:05 smithi150 ceph-mon[33096]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T19:17:05.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:05 smithi115 ceph-mon[28968]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T19:17:05.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:05 smithi115 ceph-mon[39520]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-10T19:17:05.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738415 got 34359738418 for osd.0 2024-09-10T19:17:05.462 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:06.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:06 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:06.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:06 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:06.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:06 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:07 smithi150 ceph-mon[33096]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:17:07.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:07 smithi115 ceph-mon[28968]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:17:07.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:07 smithi115 ceph-mon[39520]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:17:09.123 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:09.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:09 smithi150 ceph-mon[33096]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:09.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:09 smithi115 ceph-mon[28968]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:09.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:09 smithi115 ceph-mon[39520]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:11.180 INFO:teuthology.orchestra.run.smithi115.stdout:77309411372 2024-09-10T19:17:11.181 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[28968]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:11.181 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.181 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.181 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.181 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[39520]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.182 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:11 smithi150 ceph-mon[33096]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:11.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:11 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:11 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:11 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:11 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:11.442 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE Bus STOPPING 2024-09-10T19:17:11.762 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:17:11.763 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE Bus STOPPED 2024-09-10T19:17:11.763 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE Bus STARTING 2024-09-10T19:17:11.763 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE Serving on http://:::9283 2024-09-10T19:17:11.763 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:11 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:11] ENGINE Bus STARTED 2024-09-10T19:17:12.120 INFO:teuthology.orchestra.run.smithi115.stdout:111669149734 2024-09-10T19:17:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1789865671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:17:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/319284282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1789865671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/319284282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1789865671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:12.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/319284282' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T19:17:12.414 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:12 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[71157]: ts=2024-09-10T19:17:12.200Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000281674s 2024-09-10T19:17:12.721 INFO:teuthology.orchestra.run.smithi115.stdout:55834574895 2024-09-10T19:17:12.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149734 for osd.3 2024-09-10T19:17:12.970 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:13.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:13 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:13.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:13 smithi150 ceph-mon[33096]: Deploying daemon grafana.a on smithi150 2024-09-10T19:17:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:13 smithi150 ceph-mon[33096]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:13 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/77600331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[28968]: Deploying daemon grafana.a on smithi150 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[28968]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/77600331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[39520]: Deploying daemon grafana.a on smithi150 2024-09-10T19:17:13.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[39520]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:13.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:13 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/77600331' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T19:17:13.568 INFO:teuthology.orchestra.run.smithi115.stdout:210453397523 2024-09-10T19:17:13.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574895 for osd.1 2024-09-10T19:17:13.819 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:14 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/743772532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:17:14.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:14 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:14.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:14 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/743772532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:17:14.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:14 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:14.537 INFO:teuthology.orchestra.run.smithi115.stdout:184683593752 2024-09-10T19:17:14.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:14 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/743772532' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T19:17:14.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:14 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:15.055 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411372 for osd.2 2024-09-10T19:17:15.055 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:15.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:15 smithi115 ceph-mon[28968]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:15 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2950363116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:17:15.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:15 smithi115 ceph-mon[39520]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:15.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:15 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2950363116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:17:15.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:15 smithi150 ceph-mon[33096]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:15.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:15 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2950363116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T19:17:15.881 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397519 got 210453397523 for osd.7 2024-09-10T19:17:15.881 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:16.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:16 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:16 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:16.417 INFO:teuthology.orchestra.run.smithi115.stdout:133143986211 2024-09-10T19:17:16.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:16 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:16.669 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593748 got 184683593752 for osd.6 2024-09-10T19:17:16.669 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:17.266 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:17 smithi115 ceph-mon[28968]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:17.266 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:17 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2874135209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:17:17.266 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:17 smithi115 ceph-mon[39520]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:17.267 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:17 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2874135209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:17:17.288 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986205 got 133143986211 for osd.4 2024-09-10T19:17:17.289 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:17.632 INFO:teuthology.orchestra.run.smithi115.stdout:158913789982 2024-09-10T19:17:17.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:17 smithi150 ceph-mon[33096]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:17.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:17 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2874135209' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T19:17:18.230 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:18 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1653417731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:17:18.230 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1653417731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:17:18.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913789978 got 158913789982 for osd.5 2024-09-10T19:17:18.252 DEBUG:teuthology.parallel:result is None 2024-09-10T19:17:18.252 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T19:17:18.253 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:17:18.497 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:18.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:18 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1653417731' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T19:17:19.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:19.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:19 smithi115 ceph-mon[39520]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:19.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:19 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:19.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:19 smithi115 ceph-mon[28968]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:19.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:19 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:19.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:19 smithi150 ceph-mon[33096]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:19.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:19 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:20.309 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:17:20.311 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T19:17:20.625 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:20 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[71157]: ts=2024-09-10T19:17:20.201Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001585176s 2024-09-10T19:17:20.885 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T19:17:19.708690+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385456,"kb_used_data":6612,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575760,"statfs":{"total":767960285184,"available":765517578240,"internally_reserved":0,"allocated":6770688,"data_stored":3650326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002476"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309346+0000","last_change":"2024-09-10T19:16:03.116640+0000","last_active":"2024-09-10T19:16:09.309346+0000","last_peered":"2024-09-10T19:16:09.309346+0000","last_clean":"2024-09-10T19:16:09.309346+0000","last_became_active":"2024-09-10T19:16:03.116303+0000","last_became_peered":"2024-09-10T19:16:03.116303+0000","last_unstale":"2024-09-10T19:16:09.309346+0000","last_undegraded":"2024-09-10T19:16:09.309346+0000","last_fullsized":"2024-09-10T19:16:09.309346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:23.329001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310754+0000","last_change":"2024-09-10T19:15:56.812467+0000","last_active":"2024-09-10T19:16:09.310754+0000","last_peered":"2024-09-10T19:16:09.310754+0000","last_clean":"2024-09-10T19:16:09.310754+0000","last_became_active":"2024-09-10T19:15:56.811971+0000","last_became_peered":"2024-09-10T19:15:56.811971+0000","last_unstale":"2024-09-10T19:16:09.310754+0000","last_undegraded":"2024-09-10T19:16:09.310754+0000","last_fullsized":"2024-09-10T19:16:09.310754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:37.539871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310473+0000","last_change":"2024-09-10T19:15:58.824733+0000","last_active":"2024-09-10T19:16:09.310473+0000","last_peered":"2024-09-10T19:16:09.310473+0000","last_clean":"2024-09-10T19:16:09.310473+0000","last_became_active":"2024-09-10T19:15:58.824400+0000","last_became_peered":"2024-09-10T19:15:58.824400+0000","last_unstale":"2024-09-10T19:16:09.310473+0000","last_undegraded":"2024-09-10T19:16:09.310473+0000","last_fullsized":"2024-09-10T19:16:09.310473+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:53.747455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308265+0000","last_change":"2024-09-10T19:16:00.839328+0000","last_active":"2024-09-10T19:16:09.308265+0000","last_peered":"2024-09-10T19:16:09.308265+0000","last_clean":"2024-09-10T19:16:09.308265+0000","last_became_active":"2024-09-10T19:16:00.838755+0000","last_became_peered":"2024-09-10T19:16:00.838755+0000","last_unstale":"2024-09-10T19:16:09.308265+0000","last_undegraded":"2024-09-10T19:16:09.308265+0000","last_fullsized":"2024-09-10T19:16:09.308265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:04.387113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310929+0000","last_change":"2024-09-10T19:15:56.809353+0000","last_active":"2024-09-10T19:16:09.310929+0000","last_peered":"2024-09-10T19:16:09.310929+0000","last_clean":"2024-09-10T19:16:09.310929+0000","last_became_active":"2024-09-10T19:15:56.806738+0000","last_became_peered":"2024-09-10T19:15:56.806738+0000","last_unstale":"2024-09-10T19:16:09.310929+0000","last_undegraded":"2024-09-10T19:16:09.310929+0000","last_fullsized":"2024-09-10T19:16:09.310929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:48.924228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308273+0000","last_change":"2024-09-10T19:15:58.832461+0000","last_active":"2024-09-10T19:16:09.308273+0000","last_peered":"2024-09-10T19:16:09.308273+0000","last_clean":"2024-09-10T19:16:09.308273+0000","last_became_active":"2024-09-10T19:15:58.832157+0000","last_became_peered":"2024-09-10T19:15:58.832157+0000","last_unstale":"2024-09-10T19:16:09.308273+0000","last_undegraded":"2024-09-10T19:16:09.308273+0000","last_fullsized":"2024-09-10T19:16:09.308273+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:06.592665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679004+0000","last_change":"2024-09-10T19:16:00.831165+0000","last_active":"2024-09-10T19:16:09.679004+0000","last_peered":"2024-09-10T19:16:09.679004+0000","last_clean":"2024-09-10T19:16:09.679004+0000","last_became_active":"2024-09-10T19:16:00.830900+0000","last_became_peered":"2024-09-10T19:16:00.830900+0000","last_unstale":"2024-09-10T19:16:09.679004+0000","last_undegraded":"2024-09-10T19:16:09.679004+0000","last_fullsized":"2024-09-10T19:16:09.679004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:34.289163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308314+0000","last_change":"2024-09-10T19:16:02.841379+0000","last_active":"2024-09-10T19:16:09.308314+0000","last_peered":"2024-09-10T19:16:09.308314+0000","last_clean":"2024-09-10T19:16:09.308314+0000","last_became_active":"2024-09-10T19:16:02.841132+0000","last_became_peered":"2024-09-10T19:16:02.841132+0000","last_unstale":"2024-09-10T19:16:09.308314+0000","last_undegraded":"2024-09-10T19:16:09.308314+0000","last_fullsized":"2024-09-10T19:16:09.308314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:32.573021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680045+0000","last_change":"2024-09-10T19:15:56.811386+0000","last_active":"2024-09-10T19:16:09.680045+0000","last_peered":"2024-09-10T19:16:09.680045+0000","last_clean":"2024-09-10T19:16:09.680045+0000","last_became_active":"2024-09-10T19:15:56.811107+0000","last_became_peered":"2024-09-10T19:15:56.811107+0000","last_unstale":"2024-09-10T19:16:09.680045+0000","last_undegraded":"2024-09-10T19:16:09.680045+0000","last_fullsized":"2024-09-10T19:16:09.680045+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:34.798637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308761+0000","last_change":"2024-09-10T19:15:58.831860+0000","last_active":"2024-09-10T19:16:09.308761+0000","last_peered":"2024-09-10T19:16:09.308761+0000","last_clean":"2024-09-10T19:16:09.308761+0000","last_became_active":"2024-09-10T19:15:58.831504+0000","last_became_peered":"2024-09-10T19:15:58.831504+0000","last_unstale":"2024-09-10T19:16:09.308761+0000","last_undegraded":"2024-09-10T19:16:09.308761+0000","last_fullsized":"2024-09-10T19:16:09.308761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:49.655790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678894+0000","last_change":"2024-09-10T19:16:00.832542+0000","last_active":"2024-09-10T19:16:09.678894+0000","last_peered":"2024-09-10T19:16:09.678894+0000","last_clean":"2024-09-10T19:16:09.678894+0000","last_became_active":"2024-09-10T19:16:00.832217+0000","last_became_peered":"2024-09-10T19:16:00.832217+0000","last_unstale":"2024-09-10T19:16:09.678894+0000","last_undegraded":"2024-09-10T19:16:09.678894+0000","last_fullsized":"2024-09-10T19:16:09.678894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:37.198765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678671+0000","last_change":"2024-09-10T19:16:02.840530+0000","last_active":"2024-09-10T19:16:09.678671+0000","last_peered":"2024-09-10T19:16:09.678671+0000","last_clean":"2024-09-10T19:16:09.678671+0000","last_became_active":"2024-09-10T19:16:02.840217+0000","last_became_peered":"2024-09-10T19:16:02.840217+0000","last_unstale":"2024-09-10T19:16:09.678671+0000","last_undegraded":"2024-09-10T19:16:09.678671+0000","last_fullsized":"2024-09-10T19:16:09.678671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:08.780416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679793+0000","last_change":"2024-09-10T19:15:56.807970+0000","last_active":"2024-09-10T19:16:09.679793+0000","last_peered":"2024-09-10T19:16:09.679793+0000","last_clean":"2024-09-10T19:16:09.679793+0000","last_became_active":"2024-09-10T19:15:56.807652+0000","last_became_peered":"2024-09-10T19:15:56.807652+0000","last_unstale":"2024-09-10T19:16:09.679793+0000","last_undegraded":"2024-09-10T19:16:09.679793+0000","last_fullsized":"2024-09-10T19:16:09.679793+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:23.013672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308755+0000","last_change":"2024-09-10T19:15:58.823033+0000","last_active":"2024-09-10T19:16:09.308755+0000","last_peered":"2024-09-10T19:16:09.308755+0000","last_clean":"2024-09-10T19:16:09.308755+0000","last_became_active":"2024-09-10T19:15:58.822704+0000","last_became_peered":"2024-09-10T19:15:58.822704+0000","last_unstale":"2024-09-10T19:16:09.308755+0000","last_undegraded":"2024-09-10T19:16:09.308755+0000","last_fullsized":"2024-09-10T19:16:09.308755+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.892062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678701+0000","last_change":"2024-09-10T19:16:00.831882+0000","last_active":"2024-09-10T19:16:09.678701+0000","last_peered":"2024-09-10T19:16:09.678701+0000","last_clean":"2024-09-10T19:16:09.678701+0000","last_became_active":"2024-09-10T19:16:00.831604+0000","last_became_peered":"2024-09-10T19:16:00.831604+0000","last_unstale":"2024-09-10T19:16:09.678701+0000","last_undegraded":"2024-09-10T19:16:09.678701+0000","last_fullsized":"2024-09-10T19:16:09.678701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:09.222274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309301+0000","last_change":"2024-09-10T19:16:02.843903+0000","last_active":"2024-09-10T19:16:09.309301+0000","last_peered":"2024-09-10T19:16:09.309301+0000","last_clean":"2024-09-10T19:16:09.309301+0000","last_became_active":"2024-09-10T19:16:02.843631+0000","last_became_peered":"2024-09-10T19:16:02.843631+0000","last_unstale":"2024-09-10T19:16:09.309301+0000","last_undegraded":"2024-09-10T19:16:09.309301+0000","last_fullsized":"2024-09-10T19:16:09.309301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:22.956435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308933+0000","last_change":"2024-09-10T19:15:58.827148+0000","last_active":"2024-09-10T19:16:09.308933+0000","last_peered":"2024-09-10T19:16:09.308933+0000","last_clean":"2024-09-10T19:16:09.308933+0000","last_became_active":"2024-09-10T19:15:58.826812+0000","last_became_peered":"2024-09-10T19:15:58.826812+0000","last_unstale":"2024-09-10T19:16:09.308933+0000","last_undegraded":"2024-09-10T19:16:09.308933+0000","last_fullsized":"2024-09-10T19:16:09.308933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:54.915553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679578+0000","last_change":"2024-09-10T19:15:56.807618+0000","last_active":"2024-09-10T19:16:09.679578+0000","last_peered":"2024-09-10T19:16:09.679578+0000","last_clean":"2024-09-10T19:16:09.679578+0000","last_became_active":"2024-09-10T19:15:56.807218+0000","last_became_peered":"2024-09-10T19:15:56.807218+0000","last_unstale":"2024-09-10T19:16:09.679578+0000","last_undegraded":"2024-09-10T19:16:09.679578+0000","last_fullsized":"2024-09-10T19:16:09.679578+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:21.356289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679640+0000","last_change":"2024-09-10T19:16:00.832475+0000","last_active":"2024-09-10T19:16:09.679640+0000","last_peered":"2024-09-10T19:16:09.679640+0000","last_clean":"2024-09-10T19:16:09.679640+0000","last_became_active":"2024-09-10T19:16:00.831978+0000","last_became_peered":"2024-09-10T19:16:00.831978+0000","last_unstale":"2024-09-10T19:16:09.679640+0000","last_undegraded":"2024-09-10T19:16:09.679640+0000","last_fullsized":"2024-09-10T19:16:09.679640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:01.624513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678887+0000","last_change":"2024-09-10T19:16:02.837393+0000","last_active":"2024-09-10T19:16:09.678887+0000","last_peered":"2024-09-10T19:16:09.678887+0000","last_clean":"2024-09-10T19:16:09.678887+0000","last_became_active":"2024-09-10T19:16:02.833233+0000","last_became_peered":"2024-09-10T19:16:02.833233+0000","last_unstale":"2024-09-10T19:16:09.678887+0000","last_undegraded":"2024-09-10T19:16:09.678887+0000","last_fullsized":"2024-09-10T19:16:09.678887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:03.182381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308591+0000","last_change":"2024-09-10T19:15:58.824660+0000","last_active":"2024-09-10T19:16:09.308591+0000","last_peered":"2024-09-10T19:16:09.308591+0000","last_clean":"2024-09-10T19:16:09.308591+0000","last_became_active":"2024-09-10T19:15:58.824328+0000","last_became_peered":"2024-09-10T19:15:58.824328+0000","last_unstale":"2024-09-10T19:16:09.308591+0000","last_undegraded":"2024-09-10T19:16:09.308591+0000","last_fullsized":"2024-09-10T19:16:09.308591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:00.531536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679672+0000","last_change":"2024-09-10T19:15:56.818384+0000","last_active":"2024-09-10T19:16:09.679672+0000","last_peered":"2024-09-10T19:16:09.679672+0000","last_clean":"2024-09-10T19:16:09.679672+0000","last_became_active":"2024-09-10T19:15:56.817911+0000","last_became_peered":"2024-09-10T19:15:56.817911+0000","last_unstale":"2024-09-10T19:16:09.679672+0000","last_undegraded":"2024-09-10T19:16:09.679672+0000","last_fullsized":"2024-09-10T19:16:09.679672+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:53.905712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166971+0000","last_change":"2024-09-10T19:16:00.834040+0000","last_active":"2024-09-10T19:17:10.166971+0000","last_peered":"2024-09-10T19:17:10.166971+0000","last_clean":"2024-09-10T19:17:10.166971+0000","last_became_active":"2024-09-10T19:16:00.833736+0000","last_became_peered":"2024-09-10T19:16:00.833736+0000","last_unstale":"2024-09-10T19:17:10.166971+0000","last_undegraded":"2024-09-10T19:17:10.166971+0000","last_fullsized":"2024-09-10T19:17:10.166971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:23.714061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308664+0000","last_change":"2024-09-10T19:16:02.833914+0000","last_active":"2024-09-10T19:16:09.308664+0000","last_peered":"2024-09-10T19:16:09.308664+0000","last_clean":"2024-09-10T19:16:09.308664+0000","last_became_active":"2024-09-10T19:16:02.833627+0000","last_became_peered":"2024-09-10T19:16:02.833627+0000","last_unstale":"2024-09-10T19:16:09.308664+0000","last_undegraded":"2024-09-10T19:16:09.308664+0000","last_fullsized":"2024-09-10T19:16:09.308664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:48.704666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680104+0000","last_change":"2024-09-10T19:15:58.831933+0000","last_active":"2024-09-10T19:16:09.680104+0000","last_peered":"2024-09-10T19:16:09.680104+0000","last_clean":"2024-09-10T19:16:09.680104+0000","last_became_active":"2024-09-10T19:15:58.831679+0000","last_became_peered":"2024-09-10T19:15:58.831679+0000","last_unstale":"2024-09-10T19:16:09.680104+0000","last_undegraded":"2024-09-10T19:16:09.680104+0000","last_fullsized":"2024-09-10T19:16:09.680104+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:34.650482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679301+0000","last_change":"2024-09-10T19:15:56.820108+0000","last_active":"2024-09-10T19:16:09.679301+0000","last_peered":"2024-09-10T19:16:09.679301+0000","last_clean":"2024-09-10T19:16:09.679301+0000","last_became_active":"2024-09-10T19:15:56.818080+0000","last_became_peered":"2024-09-10T19:15:56.818080+0000","last_unstale":"2024-09-10T19:16:09.679301+0000","last_undegraded":"2024-09-10T19:16:09.679301+0000","last_fullsized":"2024-09-10T19:16:09.679301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:57.762435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167244+0000","last_change":"2024-09-10T19:16:00.836710+0000","last_active":"2024-09-10T19:17:10.167244+0000","last_peered":"2024-09-10T19:17:10.167244+0000","last_clean":"2024-09-10T19:17:10.167244+0000","last_became_active":"2024-09-10T19:16:00.836448+0000","last_became_peered":"2024-09-10T19:16:00.836448+0000","last_unstale":"2024-09-10T19:17:10.167244+0000","last_undegraded":"2024-09-10T19:17:10.167244+0000","last_fullsized":"2024-09-10T19:17:10.167244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:39.379233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680965+0000","last_change":"2024-09-10T19:16:02.840540+0000","last_active":"2024-09-10T19:16:09.680965+0000","last_peered":"2024-09-10T19:16:09.680965+0000","last_clean":"2024-09-10T19:16:09.680965+0000","last_became_active":"2024-09-10T19:16:02.840269+0000","last_became_peered":"2024-09-10T19:16:02.840269+0000","last_unstale":"2024-09-10T19:16:09.680965+0000","last_undegraded":"2024-09-10T19:16:09.680965+0000","last_fullsized":"2024-09-10T19:16:09.680965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:20.222712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308792+0000","last_change":"2024-09-10T19:15:58.827176+0000","last_active":"2024-09-10T19:16:09.308792+0000","last_peered":"2024-09-10T19:16:09.308792+0000","last_clean":"2024-09-10T19:16:09.308792+0000","last_became_active":"2024-09-10T19:15:58.826811+0000","last_became_peered":"2024-09-10T19:15:58.826811+0000","last_unstale":"2024-09-10T19:16:09.308792+0000","last_undegraded":"2024-09-10T19:16:09.308792+0000","last_fullsized":"2024-09-10T19:16:09.308792+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:13.938654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679971+0000","last_change":"2024-09-10T19:15:56.812118+0000","last_active":"2024-09-10T19:16:09.679971+0000","last_peered":"2024-09-10T19:16:09.679971+0000","last_clean":"2024-09-10T19:16:09.679971+0000","last_became_active":"2024-09-10T19:15:56.811855+0000","last_became_peered":"2024-09-10T19:15:56.811855+0000","last_unstale":"2024-09-10T19:16:09.679971+0000","last_undegraded":"2024-09-10T19:16:09.679971+0000","last_fullsized":"2024-09-10T19:16:09.679971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:22.773134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680204+0000","last_change":"2024-09-10T19:16:00.833046+0000","last_active":"2024-09-10T19:16:09.680204+0000","last_peered":"2024-09-10T19:16:09.680204+0000","last_clean":"2024-09-10T19:16:09.680204+0000","last_became_active":"2024-09-10T19:16:00.832643+0000","last_became_peered":"2024-09-10T19:16:00.832643+0000","last_unstale":"2024-09-10T19:16:09.680204+0000","last_undegraded":"2024-09-10T19:16:09.680204+0000","last_fullsized":"2024-09-10T19:16:09.680204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:07.615118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310914+0000","last_change":"2024-09-10T19:16:03.115413+0000","last_active":"2024-09-10T19:16:09.310914+0000","last_peered":"2024-09-10T19:16:09.310914+0000","last_clean":"2024-09-10T19:16:09.310914+0000","last_became_active":"2024-09-10T19:16:03.115049+0000","last_became_peered":"2024-09-10T19:16:03.115049+0000","last_unstale":"2024-09-10T19:16:09.310914+0000","last_undegraded":"2024-09-10T19:16:09.310914+0000","last_fullsized":"2024-09-10T19:16:09.310914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:02.490186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308384+0000","last_change":"2024-09-10T19:15:58.824855+0000","last_active":"2024-09-10T19:16:09.308384+0000","last_peered":"2024-09-10T19:16:09.308384+0000","last_clean":"2024-09-10T19:16:09.308384+0000","last_became_active":"2024-09-10T19:15:58.824549+0000","last_became_peered":"2024-09-10T19:15:58.824549+0000","last_unstale":"2024-09-10T19:16:09.308384+0000","last_undegraded":"2024-09-10T19:16:09.308384+0000","last_fullsized":"2024-09-10T19:16:09.308384+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:13.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681064+0000","last_change":"2024-09-10T19:15:56.809850+0000","last_active":"2024-09-10T19:16:09.681064+0000","last_peered":"2024-09-10T19:16:09.681064+0000","last_clean":"2024-09-10T19:16:09.681064+0000","last_became_active":"2024-09-10T19:15:56.809450+0000","last_became_peered":"2024-09-10T19:15:56.809450+0000","last_unstale":"2024-09-10T19:16:09.681064+0000","last_undegraded":"2024-09-10T19:16:09.681064+0000","last_fullsized":"2024-09-10T19:16:09.681064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.583399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166970+0000","last_change":"2024-09-10T19:16:00.826982+0000","last_active":"2024-09-10T19:17:10.166970+0000","last_peered":"2024-09-10T19:17:10.166970+0000","last_clean":"2024-09-10T19:17:10.166970+0000","last_became_active":"2024-09-10T19:16:00.826661+0000","last_became_peered":"2024-09-10T19:16:00.826661+0000","last_unstale":"2024-09-10T19:17:10.166970+0000","last_undegraded":"2024-09-10T19:17:10.166970+0000","last_fullsized":"2024-09-10T19:17:10.166970+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:42.772632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679835+0000","last_change":"2024-09-10T19:16:03.116078+0000","last_active":"2024-09-10T19:16:09.679835+0000","last_peered":"2024-09-10T19:16:09.679835+0000","last_clean":"2024-09-10T19:16:09.679835+0000","last_became_active":"2024-09-10T19:16:03.115796+0000","last_became_peered":"2024-09-10T19:16:03.115796+0000","last_unstale":"2024-09-10T19:16:09.679835+0000","last_undegraded":"2024-09-10T19:16:09.679835+0000","last_fullsized":"2024-09-10T19:16:09.679835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:09.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310360+0000","last_change":"2024-09-10T19:15:59.480620+0000","last_active":"2024-09-10T19:16:09.310360+0000","last_peered":"2024-09-10T19:16:09.310360+0000","last_clean":"2024-09-10T19:16:09.310360+0000","last_became_active":"2024-09-10T19:15:59.480158+0000","last_became_peered":"2024-09-10T19:15:59.480158+0000","last_unstale":"2024-09-10T19:16:09.310360+0000","last_undegraded":"2024-09-10T19:16:09.310360+0000","last_fullsized":"2024-09-10T19:16:09.310360+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:18.312669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680048+0000","last_change":"2024-09-10T19:15:56.816185+0000","last_active":"2024-09-10T19:16:09.680048+0000","last_peered":"2024-09-10T19:16:09.680048+0000","last_clean":"2024-09-10T19:16:09.680048+0000","last_became_active":"2024-09-10T19:15:56.815539+0000","last_became_peered":"2024-09-10T19:15:56.815539+0000","last_unstale":"2024-09-10T19:16:09.680048+0000","last_undegraded":"2024-09-10T19:16:09.680048+0000","last_fullsized":"2024-09-10T19:16:09.680048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:01.145410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308300+0000","last_change":"2024-09-10T19:16:00.838585+0000","last_active":"2024-09-10T19:16:09.308300+0000","last_peered":"2024-09-10T19:16:09.308300+0000","last_clean":"2024-09-10T19:16:09.308300+0000","last_became_active":"2024-09-10T19:16:00.838141+0000","last_became_peered":"2024-09-10T19:16:00.838141+0000","last_unstale":"2024-09-10T19:16:09.308300+0000","last_undegraded":"2024-09-10T19:16:09.308300+0000","last_fullsized":"2024-09-10T19:16:09.308300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.366436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308676+0000","last_change":"2024-09-10T19:16:02.831571+0000","last_active":"2024-09-10T19:16:09.308676+0000","last_peered":"2024-09-10T19:16:09.308676+0000","last_clean":"2024-09-10T19:16:09.308676+0000","last_became_active":"2024-09-10T19:16:02.831267+0000","last_became_peered":"2024-09-10T19:16:02.831267+0000","last_unstale":"2024-09-10T19:16:09.308676+0000","last_undegraded":"2024-09-10T19:16:09.308676+0000","last_fullsized":"2024-09-10T19:16:09.308676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:43.395500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311422+0000","last_change":"2024-09-10T19:15:59.477546+0000","last_active":"2024-09-10T19:16:09.311422+0000","last_peered":"2024-09-10T19:16:09.311422+0000","last_clean":"2024-09-10T19:16:09.311422+0000","last_became_active":"2024-09-10T19:15:59.477251+0000","last_became_peered":"2024-09-10T19:15:59.477251+0000","last_unstale":"2024-09-10T19:16:09.311422+0000","last_undegraded":"2024-09-10T19:16:09.311422+0000","last_fullsized":"2024-09-10T19:16:09.311422+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:52.122774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680114+0000","last_change":"2024-09-10T19:15:56.808503+0000","last_active":"2024-09-10T19:16:09.680114+0000","last_peered":"2024-09-10T19:16:09.680114+0000","last_clean":"2024-09-10T19:16:09.680114+0000","last_became_active":"2024-09-10T19:15:56.808237+0000","last_became_peered":"2024-09-10T19:15:56.808237+0000","last_unstale":"2024-09-10T19:16:09.680114+0000","last_undegraded":"2024-09-10T19:16:09.680114+0000","last_fullsized":"2024-09-10T19:16:09.680114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:08.346896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680692+0000","last_change":"2024-09-10T19:16:00.838984+0000","last_active":"2024-09-10T19:16:09.680692+0000","last_peered":"2024-09-10T19:16:09.680692+0000","last_clean":"2024-09-10T19:16:09.680692+0000","last_became_active":"2024-09-10T19:16:00.837939+0000","last_became_peered":"2024-09-10T19:16:00.837939+0000","last_unstale":"2024-09-10T19:16:09.680692+0000","last_undegraded":"2024-09-10T19:16:09.680692+0000","last_fullsized":"2024-09-10T19:16:09.680692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:34.842017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678642+0000","last_change":"2024-09-10T19:16:03.115446+0000","last_active":"2024-09-10T19:16:09.678642+0000","last_peered":"2024-09-10T19:16:09.678642+0000","last_clean":"2024-09-10T19:16:09.678642+0000","last_became_active":"2024-09-10T19:16:03.114988+0000","last_became_peered":"2024-09-10T19:16:03.114988+0000","last_unstale":"2024-09-10T19:16:09.678642+0000","last_undegraded":"2024-09-10T19:16:09.678642+0000","last_fullsized":"2024-09-10T19:16:09.678642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:16.052478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679550+0000","last_change":"2024-09-10T19:15:58.828850+0000","last_active":"2024-09-10T19:16:09.679550+0000","last_peered":"2024-09-10T19:16:09.679550+0000","last_clean":"2024-09-10T19:16:09.679550+0000","last_became_active":"2024-09-10T19:15:58.828476+0000","last_became_peered":"2024-09-10T19:15:58.828476+0000","last_unstale":"2024-09-10T19:16:09.679550+0000","last_undegraded":"2024-09-10T19:16:09.679550+0000","last_fullsized":"2024-09-10T19:16:09.679550+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:10.357022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679797+0000","last_change":"2024-09-10T19:15:56.816109+0000","last_active":"2024-09-10T19:16:09.679797+0000","last_peered":"2024-09-10T19:16:09.679797+0000","last_clean":"2024-09-10T19:16:09.679797+0000","last_became_active":"2024-09-10T19:15:56.815830+0000","last_became_peered":"2024-09-10T19:15:56.815830+0000","last_unstale":"2024-09-10T19:16:09.679797+0000","last_undegraded":"2024-09-10T19:16:09.679797+0000","last_fullsized":"2024-09-10T19:16:09.679797+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:40.353434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166971+0000","last_change":"2024-09-10T19:16:00.834636+0000","last_active":"2024-09-10T19:17:10.166971+0000","last_peered":"2024-09-10T19:17:10.166971+0000","last_clean":"2024-09-10T19:17:10.166971+0000","last_became_active":"2024-09-10T19:16:00.834150+0000","last_became_peered":"2024-09-10T19:16:00.834150+0000","last_unstale":"2024-09-10T19:17:10.166971+0000","last_undegraded":"2024-09-10T19:17:10.166971+0000","last_fullsized":"2024-09-10T19:17:10.166971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:04.114396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309145+0000","last_change":"2024-09-10T19:16:02.842367+0000","last_active":"2024-09-10T19:16:09.309145+0000","last_peered":"2024-09-10T19:16:09.309145+0000","last_clean":"2024-09-10T19:16:09.309145+0000","last_became_active":"2024-09-10T19:16:02.842061+0000","last_became_peered":"2024-09-10T19:16:02.842061+0000","last_unstale":"2024-09-10T19:16:09.309145+0000","last_undegraded":"2024-09-10T19:16:09.309145+0000","last_fullsized":"2024-09-10T19:16:09.309145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:34.471651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":80,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166972+0000","last_change":"2024-09-10T19:15:58.835178+0000","last_active":"2024-09-10T19:17:10.166972+0000","last_peered":"2024-09-10T19:17:10.166972+0000","last_clean":"2024-09-10T19:17:10.166972+0000","last_became_active":"2024-09-10T19:15:58.834893+0000","last_became_peered":"2024-09-10T19:15:58.834893+0000","last_unstale":"2024-09-10T19:17:10.166972+0000","last_undegraded":"2024-09-10T19:17:10.166972+0000","last_fullsized":"2024-09-10T19:17:10.166972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:36.797750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680691+0000","last_change":"2024-09-10T19:15:56.814418+0000","last_active":"2024-09-10T19:16:09.680691+0000","last_peered":"2024-09-10T19:16:09.680691+0000","last_clean":"2024-09-10T19:16:09.680691+0000","last_became_active":"2024-09-10T19:15:56.813534+0000","last_became_peered":"2024-09-10T19:15:56.813534+0000","last_unstale":"2024-09-10T19:16:09.680691+0000","last_undegraded":"2024-09-10T19:16:09.680691+0000","last_fullsized":"2024-09-10T19:16:09.680691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:32.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309723+0000","last_change":"2024-09-10T19:16:00.832390+0000","last_active":"2024-09-10T19:16:09.309723+0000","last_peered":"2024-09-10T19:16:09.309723+0000","last_clean":"2024-09-10T19:16:09.309723+0000","last_became_active":"2024-09-10T19:16:00.832129+0000","last_became_peered":"2024-09-10T19:16:00.832129+0000","last_unstale":"2024-09-10T19:16:09.309723+0000","last_undegraded":"2024-09-10T19:16:09.309723+0000","last_fullsized":"2024-09-10T19:16:09.309723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:51.232485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310441+0000","last_change":"2024-09-10T19:16:02.832266+0000","last_active":"2024-09-10T19:16:09.310441+0000","last_peered":"2024-09-10T19:16:09.310441+0000","last_clean":"2024-09-10T19:16:09.310441+0000","last_became_active":"2024-09-10T19:16:02.830372+0000","last_became_peered":"2024-09-10T19:16:02.830372+0000","last_unstale":"2024-09-10T19:16:09.310441+0000","last_undegraded":"2024-09-10T19:16:09.310441+0000","last_fullsized":"2024-09-10T19:16:09.310441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:53.660233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680116+0000","last_change":"2024-09-10T19:15:58.829728+0000","last_active":"2024-09-10T19:16:09.680116+0000","last_peered":"2024-09-10T19:16:09.680116+0000","last_clean":"2024-09-10T19:16:09.680116+0000","last_became_active":"2024-09-10T19:15:58.829395+0000","last_became_peered":"2024-09-10T19:15:58.829395+0000","last_unstale":"2024-09-10T19:16:09.680116+0000","last_undegraded":"2024-09-10T19:16:09.680116+0000","last_fullsized":"2024-09-10T19:16:09.680116+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:05.159670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'5","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:17.969263+0000","last_change":"2024-09-10T19:16:00.820995+0000","last_active":"2024-09-10T19:17:17.969263+0000","last_peered":"2024-09-10T19:17:17.969263+0000","last_clean":"2024-09-10T19:17:17.969263+0000","last_became_active":"2024-09-10T19:15:56.816813+0000","last_became_peered":"2024-09-10T19:15:56.816813+0000","last_unstale":"2024-09-10T19:17:17.969263+0000","last_undegraded":"2024-09-10T19:17:17.969263+0000","last_fullsized":"2024-09-10T19:17:17.969263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:01.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033535900000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678418+0000","last_change":"2024-09-10T19:15:56.805620+0000","last_active":"2024-09-10T19:16:09.678418+0000","last_peered":"2024-09-10T19:16:09.678418+0000","last_clean":"2024-09-10T19:16:09.678418+0000","last_became_active":"2024-09-10T19:15:56.805336+0000","last_became_peered":"2024-09-10T19:15:56.805336+0000","last_unstale":"2024-09-10T19:16:09.678418+0000","last_undegraded":"2024-09-10T19:16:09.678418+0000","last_fullsized":"2024-09-10T19:16:09.678418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:13.400160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678783+0000","last_change":"2024-09-10T19:16:00.831347+0000","last_active":"2024-09-10T19:16:09.678783+0000","last_peered":"2024-09-10T19:16:09.678783+0000","last_clean":"2024-09-10T19:16:09.678783+0000","last_became_active":"2024-09-10T19:16:00.831001+0000","last_became_peered":"2024-09-10T19:16:00.831001+0000","last_unstale":"2024-09-10T19:16:09.678783+0000","last_undegraded":"2024-09-10T19:16:09.678783+0000","last_fullsized":"2024-09-10T19:16:09.678783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:49.082418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679035+0000","last_change":"2024-09-10T19:16:03.116809+0000","last_active":"2024-09-10T19:16:09.679035+0000","last_peered":"2024-09-10T19:16:09.679035+0000","last_clean":"2024-09-10T19:16:09.679035+0000","last_became_active":"2024-09-10T19:16:03.116497+0000","last_became_peered":"2024-09-10T19:16:03.116497+0000","last_unstale":"2024-09-10T19:16:09.679035+0000","last_undegraded":"2024-09-10T19:16:09.679035+0000","last_fullsized":"2024-09-10T19:16:09.679035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:48.716725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311220+0000","last_change":"2024-09-10T19:15:58.830601+0000","last_active":"2024-09-10T19:16:09.311220+0000","last_peered":"2024-09-10T19:16:09.311220+0000","last_clean":"2024-09-10T19:16:09.311220+0000","last_became_active":"2024-09-10T19:15:58.830343+0000","last_became_peered":"2024-09-10T19:15:58.830343+0000","last_unstale":"2024-09-10T19:16:09.311220+0000","last_undegraded":"2024-09-10T19:16:09.311220+0000","last_fullsized":"2024-09-10T19:16:09.311220+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:15.556782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309724+0000","last_change":"2024-09-10T19:16:00.825660+0000","last_active":"2024-09-10T19:16:09.309724+0000","last_peered":"2024-09-10T19:16:09.309724+0000","last_clean":"2024-09-10T19:16:09.309724+0000","last_became_active":"2024-09-10T19:15:56.813686+0000","last_became_peered":"2024-09-10T19:15:56.813686+0000","last_unstale":"2024-09-10T19:16:09.309724+0000","last_undegraded":"2024-09-10T19:16:09.309724+0000","last_fullsized":"2024-09-10T19:16:09.309724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:51.761088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080658900000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679425+0000","last_change":"2024-09-10T19:15:56.818639+0000","last_active":"2024-09-10T19:16:09.679425+0000","last_peered":"2024-09-10T19:16:09.679425+0000","last_clean":"2024-09-10T19:16:09.679425+0000","last_became_active":"2024-09-10T19:15:56.818379+0000","last_became_peered":"2024-09-10T19:15:56.818379+0000","last_unstale":"2024-09-10T19:16:09.679425+0000","last_undegraded":"2024-09-10T19:16:09.679425+0000","last_fullsized":"2024-09-10T19:16:09.679425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:08.135051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681019+0000","last_change":"2024-09-10T19:16:00.836434+0000","last_active":"2024-09-10T19:16:09.681019+0000","last_peered":"2024-09-10T19:16:09.681019+0000","last_clean":"2024-09-10T19:16:09.681019+0000","last_became_active":"2024-09-10T19:16:00.836171+0000","last_became_peered":"2024-09-10T19:16:00.836171+0000","last_unstale":"2024-09-10T19:16:09.681019+0000","last_undegraded":"2024-09-10T19:16:09.681019+0000","last_fullsized":"2024-09-10T19:16:09.681019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:15.914055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679554+0000","last_change":"2024-09-10T19:16:02.839239+0000","last_active":"2024-09-10T19:16:09.679554+0000","last_peered":"2024-09-10T19:16:09.679554+0000","last_clean":"2024-09-10T19:16:09.679554+0000","last_became_active":"2024-09-10T19:16:02.838960+0000","last_became_peered":"2024-09-10T19:16:02.838960+0000","last_unstale":"2024-09-10T19:16:09.679554+0000","last_undegraded":"2024-09-10T19:16:09.679554+0000","last_fullsized":"2024-09-10T19:16:09.679554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:39.695018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679740+0000","last_change":"2024-09-10T19:15:59.479170+0000","last_active":"2024-09-10T19:16:09.679740+0000","last_peered":"2024-09-10T19:16:09.679740+0000","last_clean":"2024-09-10T19:16:09.679740+0000","last_became_active":"2024-09-10T19:15:59.478866+0000","last_became_peered":"2024-09-10T19:15:59.478866+0000","last_unstale":"2024-09-10T19:16:09.679740+0000","last_undegraded":"2024-09-10T19:16:09.679740+0000","last_fullsized":"2024-09-10T19:16:09.679740+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:05.509410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310489+0000","last_change":"2024-09-10T19:16:00.823451+0000","last_active":"2024-09-10T19:16:09.310489+0000","last_peered":"2024-09-10T19:16:09.310489+0000","last_clean":"2024-09-10T19:16:09.310489+0000","last_became_active":"2024-09-10T19:15:56.812606+0000","last_became_peered":"2024-09-10T19:15:56.812606+0000","last_unstale":"2024-09-10T19:16:09.310489+0000","last_undegraded":"2024-09-10T19:16:09.310489+0000","last_fullsized":"2024-09-10T19:16:09.310489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:04.194548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030027740000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679251+0000","last_change":"2024-09-10T19:15:56.809970+0000","last_active":"2024-09-10T19:16:09.679251+0000","last_peered":"2024-09-10T19:16:09.679251+0000","last_clean":"2024-09-10T19:16:09.679251+0000","last_became_active":"2024-09-10T19:15:56.809700+0000","last_became_peered":"2024-09-10T19:15:56.809700+0000","last_unstale":"2024-09-10T19:16:09.679251+0000","last_undegraded":"2024-09-10T19:16:09.679251+0000","last_fullsized":"2024-09-10T19:16:09.679251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:56.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:16:35.751619+0000","last_change":"2024-09-10T19:15:46.309327+0000","last_active":"2024-09-10T19:16:35.751619+0000","last_peered":"2024-09-10T19:16:35.751619+0000","last_clean":"2024-09-10T19:16:35.751619+0000","last_became_active":"2024-09-10T19:15:46.304277+0000","last_became_peered":"2024-09-10T19:15:46.304277+0000","last_unstale":"2024-09-10T19:16:35.751619+0000","last_undegraded":"2024-09-10T19:16:35.751619+0000","last_fullsized":"2024-09-10T19:16:35.751619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:13:50.198934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:13:50.198934+0000","last_clean_scrub_stamp":"2024-09-10T19:13:50.198934+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:16.862540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679546+0000","last_change":"2024-09-10T19:16:00.834559+0000","last_active":"2024-09-10T19:16:09.679546+0000","last_peered":"2024-09-10T19:16:09.679546+0000","last_clean":"2024-09-10T19:16:09.679546+0000","last_became_active":"2024-09-10T19:16:00.833709+0000","last_became_peered":"2024-09-10T19:16:00.833709+0000","last_unstale":"2024-09-10T19:16:09.679546+0000","last_undegraded":"2024-09-10T19:16:09.679546+0000","last_fullsized":"2024-09-10T19:16:09.679546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:00.147941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680562+0000","last_change":"2024-09-10T19:16:02.836722+0000","last_active":"2024-09-10T19:16:09.680562+0000","last_peered":"2024-09-10T19:16:09.680562+0000","last_clean":"2024-09-10T19:16:09.680562+0000","last_became_active":"2024-09-10T19:16:02.836420+0000","last_became_peered":"2024-09-10T19:16:02.836420+0000","last_unstale":"2024-09-10T19:16:09.680562+0000","last_undegraded":"2024-09-10T19:16:09.680562+0000","last_fullsized":"2024-09-10T19:16:09.680562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:00.547439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680914+0000","last_change":"2024-09-10T19:15:59.478109+0000","last_active":"2024-09-10T19:16:09.680914+0000","last_peered":"2024-09-10T19:16:09.680914+0000","last_clean":"2024-09-10T19:16:09.680914+0000","last_became_active":"2024-09-10T19:15:59.477833+0000","last_became_peered":"2024-09-10T19:15:59.477833+0000","last_unstale":"2024-09-10T19:16:09.680914+0000","last_undegraded":"2024-09-10T19:16:09.680914+0000","last_fullsized":"2024-09-10T19:16:09.680914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:25.856436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679224+0000","last_change":"2024-09-10T19:15:56.807617+0000","last_active":"2024-09-10T19:16:09.679224+0000","last_peered":"2024-09-10T19:16:09.679224+0000","last_clean":"2024-09-10T19:16:09.679224+0000","last_became_active":"2024-09-10T19:15:56.807350+0000","last_became_peered":"2024-09-10T19:15:56.807350+0000","last_unstale":"2024-09-10T19:16:09.679224+0000","last_undegraded":"2024-09-10T19:16:09.679224+0000","last_fullsized":"2024-09-10T19:16:09.679224+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:44.422976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678604+0000","last_change":"2024-09-10T19:16:00.835608+0000","last_active":"2024-09-10T19:16:09.678604+0000","last_peered":"2024-09-10T19:16:09.678604+0000","last_clean":"2024-09-10T19:16:09.678604+0000","last_became_active":"2024-09-10T19:16:00.835272+0000","last_became_peered":"2024-09-10T19:16:00.835272+0000","last_unstale":"2024-09-10T19:16:09.678604+0000","last_undegraded":"2024-09-10T19:16:09.678604+0000","last_fullsized":"2024-09-10T19:16:09.678604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:49.984604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679500+0000","last_change":"2024-09-10T19:16:02.839251+0000","last_active":"2024-09-10T19:16:09.679500+0000","last_peered":"2024-09-10T19:16:09.679500+0000","last_clean":"2024-09-10T19:16:09.679500+0000","last_became_active":"2024-09-10T19:16:02.839050+0000","last_became_peered":"2024-09-10T19:16:02.839050+0000","last_unstale":"2024-09-10T19:16:09.679500+0000","last_undegraded":"2024-09-10T19:16:09.679500+0000","last_fullsized":"2024-09-10T19:16:09.679500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:38.771883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311307+0000","last_change":"2024-09-10T19:15:58.831395+0000","last_active":"2024-09-10T19:16:09.311307+0000","last_peered":"2024-09-10T19:16:09.311307+0000","last_clean":"2024-09-10T19:16:09.311307+0000","last_became_active":"2024-09-10T19:15:58.831143+0000","last_became_peered":"2024-09-10T19:15:58.831143+0000","last_unstale":"2024-09-10T19:16:09.311307+0000","last_undegraded":"2024-09-10T19:16:09.311307+0000","last_fullsized":"2024-09-10T19:16:09.311307+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:50.024094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680091+0000","last_change":"2024-09-10T19:15:56.818922+0000","last_active":"2024-09-10T19:16:09.680091+0000","last_peered":"2024-09-10T19:16:09.680091+0000","last_clean":"2024-09-10T19:16:09.680091+0000","last_became_active":"2024-09-10T19:15:56.818614+0000","last_became_peered":"2024-09-10T19:15:56.818614+0000","last_unstale":"2024-09-10T19:16:09.680091+0000","last_undegraded":"2024-09-10T19:16:09.680091+0000","last_fullsized":"2024-09-10T19:16:09.680091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:05.135056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678469+0000","last_change":"2024-09-10T19:16:00.840244+0000","last_active":"2024-09-10T19:16:09.678469+0000","last_peered":"2024-09-10T19:16:09.678469+0000","last_clean":"2024-09-10T19:16:09.678469+0000","last_became_active":"2024-09-10T19:16:00.839917+0000","last_became_peered":"2024-09-10T19:16:00.839917+0000","last_unstale":"2024-09-10T19:16:09.678469+0000","last_undegraded":"2024-09-10T19:16:09.678469+0000","last_fullsized":"2024-09-10T19:16:09.678469+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:10.279003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309530+0000","last_change":"2024-09-10T19:16:02.841310+0000","last_active":"2024-09-10T19:16:09.309530+0000","last_peered":"2024-09-10T19:16:09.309530+0000","last_clean":"2024-09-10T19:16:09.309530+0000","last_became_active":"2024-09-10T19:16:02.840875+0000","last_became_peered":"2024-09-10T19:16:02.840875+0000","last_unstale":"2024-09-10T19:16:09.309530+0000","last_undegraded":"2024-09-10T19:16:09.309530+0000","last_fullsized":"2024-09-10T19:16:09.309530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:08.582420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680703+0000","last_change":"2024-09-10T19:15:59.478048+0000","last_active":"2024-09-10T19:16:09.680703+0000","last_peered":"2024-09-10T19:16:09.680703+0000","last_clean":"2024-09-10T19:16:09.680703+0000","last_became_active":"2024-09-10T19:15:59.477665+0000","last_became_peered":"2024-09-10T19:15:59.477665+0000","last_unstale":"2024-09-10T19:16:09.680703+0000","last_undegraded":"2024-09-10T19:16:09.680703+0000","last_fullsized":"2024-09-10T19:16:09.680703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:26.018844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678909+0000","last_change":"2024-09-10T19:15:56.810583+0000","last_active":"2024-09-10T19:16:09.678909+0000","last_peered":"2024-09-10T19:16:09.678909+0000","last_clean":"2024-09-10T19:16:09.678909+0000","last_became_active":"2024-09-10T19:15:56.810308+0000","last_became_peered":"2024-09-10T19:15:56.810308+0000","last_unstale":"2024-09-10T19:16:09.678909+0000","last_undegraded":"2024-09-10T19:16:09.678909+0000","last_fullsized":"2024-09-10T19:16:09.678909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:58.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167209+0000","last_change":"2024-09-10T19:16:00.832364+0000","last_active":"2024-09-10T19:17:10.167209+0000","last_peered":"2024-09-10T19:17:10.167209+0000","last_clean":"2024-09-10T19:17:10.167209+0000","last_became_active":"2024-09-10T19:16:00.832040+0000","last_became_peered":"2024-09-10T19:16:00.832040+0000","last_unstale":"2024-09-10T19:17:10.167209+0000","last_undegraded":"2024-09-10T19:17:10.167209+0000","last_fullsized":"2024-09-10T19:17:10.167209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:50.602033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680147+0000","last_change":"2024-09-10T19:16:03.115997+0000","last_active":"2024-09-10T19:16:09.680147+0000","last_peered":"2024-09-10T19:16:09.680147+0000","last_clean":"2024-09-10T19:16:09.680147+0000","last_became_active":"2024-09-10T19:16:03.115726+0000","last_became_peered":"2024-09-10T19:16:03.115726+0000","last_unstale":"2024-09-10T19:16:09.680147+0000","last_undegraded":"2024-09-10T19:16:09.680147+0000","last_fullsized":"2024-09-10T19:16:09.680147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:43.776994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308390+0000","last_change":"2024-09-10T19:15:58.830722+0000","last_active":"2024-09-10T19:16:09.308390+0000","last_peered":"2024-09-10T19:16:09.308390+0000","last_clean":"2024-09-10T19:16:09.308390+0000","last_became_active":"2024-09-10T19:15:58.830401+0000","last_became_peered":"2024-09-10T19:15:58.830401+0000","last_unstale":"2024-09-10T19:16:09.308390+0000","last_undegraded":"2024-09-10T19:16:09.308390+0000","last_fullsized":"2024-09-10T19:16:09.308390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:28.117209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308985+0000","last_change":"2024-09-10T19:15:56.806373+0000","last_active":"2024-09-10T19:16:09.308985+0000","last_peered":"2024-09-10T19:16:09.308985+0000","last_clean":"2024-09-10T19:16:09.308985+0000","last_became_active":"2024-09-10T19:15:56.806129+0000","last_became_peered":"2024-09-10T19:15:56.806129+0000","last_unstale":"2024-09-10T19:16:09.308985+0000","last_undegraded":"2024-09-10T19:16:09.308985+0000","last_fullsized":"2024-09-10T19:16:09.308985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:21.665650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678345+0000","last_change":"2024-09-10T19:16:00.831974+0000","last_active":"2024-09-10T19:16:09.678345+0000","last_peered":"2024-09-10T19:16:09.678345+0000","last_clean":"2024-09-10T19:16:09.678345+0000","last_became_active":"2024-09-10T19:16:00.831691+0000","last_became_peered":"2024-09-10T19:16:00.831691+0000","last_unstale":"2024-09-10T19:16:09.678345+0000","last_undegraded":"2024-09-10T19:16:09.678345+0000","last_fullsized":"2024-09-10T19:16:09.678345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:03.901713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309514+0000","last_change":"2024-09-10T19:16:02.839599+0000","last_active":"2024-09-10T19:16:09.309514+0000","last_peered":"2024-09-10T19:16:09.309514+0000","last_clean":"2024-09-10T19:16:09.309514+0000","last_became_active":"2024-09-10T19:16:02.838951+0000","last_became_peered":"2024-09-10T19:16:02.838951+0000","last_unstale":"2024-09-10T19:16:09.309514+0000","last_undegraded":"2024-09-10T19:16:09.309514+0000","last_fullsized":"2024-09-10T19:16:09.309514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:05.429846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311475+0000","last_change":"2024-09-10T19:15:59.477764+0000","last_active":"2024-09-10T19:16:09.311475+0000","last_peered":"2024-09-10T19:16:09.311475+0000","last_clean":"2024-09-10T19:16:09.311475+0000","last_became_active":"2024-09-10T19:15:59.477508+0000","last_became_peered":"2024-09-10T19:15:59.477508+0000","last_unstale":"2024-09-10T19:16:09.311475+0000","last_undegraded":"2024-09-10T19:16:09.311475+0000","last_fullsized":"2024-09-10T19:16:09.311475+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:01.854980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678583+0000","last_change":"2024-09-10T19:15:56.811444+0000","last_active":"2024-09-10T19:16:09.678583+0000","last_peered":"2024-09-10T19:16:09.678583+0000","last_clean":"2024-09-10T19:16:09.678583+0000","last_became_active":"2024-09-10T19:15:56.811188+0000","last_became_peered":"2024-09-10T19:15:56.811188+0000","last_unstale":"2024-09-10T19:16:09.678583+0000","last_undegraded":"2024-09-10T19:16:09.678583+0000","last_fullsized":"2024-09-10T19:16:09.678583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:16.906924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310898+0000","last_change":"2024-09-10T19:16:00.825360+0000","last_active":"2024-09-10T19:16:09.310898+0000","last_peered":"2024-09-10T19:16:09.310898+0000","last_clean":"2024-09-10T19:16:09.310898+0000","last_became_active":"2024-09-10T19:16:00.825076+0000","last_became_peered":"2024-09-10T19:16:00.825076+0000","last_unstale":"2024-09-10T19:16:09.310898+0000","last_undegraded":"2024-09-10T19:16:09.310898+0000","last_fullsized":"2024-09-10T19:16:09.310898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:22.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678724+0000","last_change":"2024-09-10T19:16:02.836433+0000","last_active":"2024-09-10T19:16:09.678724+0000","last_peered":"2024-09-10T19:16:09.678724+0000","last_clean":"2024-09-10T19:16:09.678724+0000","last_became_active":"2024-09-10T19:16:02.836159+0000","last_became_peered":"2024-09-10T19:16:02.836159+0000","last_unstale":"2024-09-10T19:16:09.678724+0000","last_undegraded":"2024-09-10T19:16:09.678724+0000","last_fullsized":"2024-09-10T19:16:09.678724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:09.020929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678996+0000","last_change":"2024-09-10T19:15:56.807417+0000","last_active":"2024-09-10T19:16:09.678996+0000","last_peered":"2024-09-10T19:16:09.678996+0000","last_clean":"2024-09-10T19:16:09.678996+0000","last_became_active":"2024-09-10T19:15:56.807162+0000","last_became_peered":"2024-09-10T19:15:56.807162+0000","last_unstale":"2024-09-10T19:16:09.678996+0000","last_undegraded":"2024-09-10T19:16:09.678996+0000","last_fullsized":"2024-09-10T19:16:09.678996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:47.474664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311049+0000","last_change":"2024-09-10T19:15:59.477898+0000","last_active":"2024-09-10T19:16:09.311049+0000","last_peered":"2024-09-10T19:16:09.311049+0000","last_clean":"2024-09-10T19:16:09.311049+0000","last_became_active":"2024-09-10T19:15:59.477405+0000","last_became_peered":"2024-09-10T19:15:59.477405+0000","last_unstale":"2024-09-10T19:16:09.311049+0000","last_undegraded":"2024-09-10T19:16:09.311049+0000","last_fullsized":"2024-09-10T19:16:09.311049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:40.664642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679557+0000","last_change":"2024-09-10T19:16:00.837493+0000","last_active":"2024-09-10T19:16:09.679557+0000","last_peered":"2024-09-10T19:16:09.679557+0000","last_clean":"2024-09-10T19:16:09.679557+0000","last_became_active":"2024-09-10T19:16:00.837210+0000","last_became_peered":"2024-09-10T19:16:00.837210+0000","last_unstale":"2024-09-10T19:16:09.679557+0000","last_undegraded":"2024-09-10T19:16:09.679557+0000","last_fullsized":"2024-09-10T19:16:09.679557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:43.335638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678952+0000","last_change":"2024-09-10T19:16:03.116153+0000","last_active":"2024-09-10T19:16:09.678952+0000","last_peered":"2024-09-10T19:16:09.678952+0000","last_clean":"2024-09-10T19:16:09.678952+0000","last_became_active":"2024-09-10T19:16:03.115871+0000","last_became_peered":"2024-09-10T19:16:03.115871+0000","last_unstale":"2024-09-10T19:16:09.678952+0000","last_undegraded":"2024-09-10T19:16:09.678952+0000","last_fullsized":"2024-09-10T19:16:09.678952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:21.986445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311160+0000","last_change":"2024-09-10T19:15:59.480380+0000","last_active":"2024-09-10T19:16:09.311160+0000","last_peered":"2024-09-10T19:16:09.311160+0000","last_clean":"2024-09-10T19:16:09.311160+0000","last_became_active":"2024-09-10T19:15:59.480104+0000","last_became_peered":"2024-09-10T19:15:59.480104+0000","last_unstale":"2024-09-10T19:16:09.311160+0000","last_undegraded":"2024-09-10T19:16:09.311160+0000","last_fullsized":"2024-09-10T19:16:09.311160+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:18.922754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680621+0000","last_change":"2024-09-10T19:15:56.814668+0000","last_active":"2024-09-10T19:16:09.680621+0000","last_peered":"2024-09-10T19:16:09.680621+0000","last_clean":"2024-09-10T19:16:09.680621+0000","last_became_active":"2024-09-10T19:15:56.813933+0000","last_became_peered":"2024-09-10T19:15:56.813933+0000","last_unstale":"2024-09-10T19:16:09.680621+0000","last_undegraded":"2024-09-10T19:16:09.680621+0000","last_fullsized":"2024-09-10T19:16:09.680621+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:51.925652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167231+0000","last_change":"2024-09-10T19:16:00.837931+0000","last_active":"2024-09-10T19:17:10.167231+0000","last_peered":"2024-09-10T19:17:10.167231+0000","last_clean":"2024-09-10T19:17:10.167231+0000","last_became_active":"2024-09-10T19:16:00.837619+0000","last_became_peered":"2024-09-10T19:16:00.837619+0000","last_unstale":"2024-09-10T19:17:10.167231+0000","last_undegraded":"2024-09-10T19:17:10.167231+0000","last_fullsized":"2024-09-10T19:17:10.167231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:53.040495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310850+0000","last_change":"2024-09-10T19:16:02.842576+0000","last_active":"2024-09-10T19:16:09.310850+0000","last_peered":"2024-09-10T19:16:09.310850+0000","last_clean":"2024-09-10T19:16:09.310850+0000","last_became_active":"2024-09-10T19:16:02.842284+0000","last_became_peered":"2024-09-10T19:16:02.842284+0000","last_unstale":"2024-09-10T19:16:09.310850+0000","last_undegraded":"2024-09-10T19:16:09.310850+0000","last_fullsized":"2024-09-10T19:16:09.310850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:25.991343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680877+0000","last_change":"2024-09-10T19:15:59.480057+0000","last_active":"2024-09-10T19:16:09.680877+0000","last_peered":"2024-09-10T19:16:09.680877+0000","last_clean":"2024-09-10T19:16:09.680877+0000","last_became_active":"2024-09-10T19:15:59.479711+0000","last_became_peered":"2024-09-10T19:15:59.479711+0000","last_unstale":"2024-09-10T19:16:09.680877+0000","last_undegraded":"2024-09-10T19:16:09.680877+0000","last_fullsized":"2024-09-10T19:16:09.680877+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:46.572648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311530+0000","last_change":"2024-09-10T19:15:56.814802+0000","last_active":"2024-09-10T19:16:09.311530+0000","last_peered":"2024-09-10T19:16:09.311530+0000","last_clean":"2024-09-10T19:16:09.311530+0000","last_became_active":"2024-09-10T19:15:56.814329+0000","last_became_peered":"2024-09-10T19:15:56.814329+0000","last_unstale":"2024-09-10T19:16:09.311530+0000","last_undegraded":"2024-09-10T19:16:09.311530+0000","last_fullsized":"2024-09-10T19:16:09.311530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:47.101659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167039+0000","last_change":"2024-09-10T19:16:00.836128+0000","last_active":"2024-09-10T19:17:10.167039+0000","last_peered":"2024-09-10T19:17:10.167039+0000","last_clean":"2024-09-10T19:17:10.167039+0000","last_became_active":"2024-09-10T19:16:00.835790+0000","last_became_peered":"2024-09-10T19:16:00.835790+0000","last_unstale":"2024-09-10T19:17:10.167039+0000","last_undegraded":"2024-09-10T19:17:10.167039+0000","last_fullsized":"2024-09-10T19:17:10.167039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:48.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308663+0000","last_change":"2024-09-10T19:16:02.832727+0000","last_active":"2024-09-10T19:16:09.308663+0000","last_peered":"2024-09-10T19:16:09.308663+0000","last_clean":"2024-09-10T19:16:09.308663+0000","last_became_active":"2024-09-10T19:16:02.832469+0000","last_became_peered":"2024-09-10T19:16:02.832469+0000","last_unstale":"2024-09-10T19:16:09.308663+0000","last_undegraded":"2024-09-10T19:16:09.308663+0000","last_fullsized":"2024-09-10T19:16:09.308663+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:41.568602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679532+0000","last_change":"2024-09-10T19:15:58.835886+0000","last_active":"2024-09-10T19:16:09.679532+0000","last_peered":"2024-09-10T19:16:09.679532+0000","last_clean":"2024-09-10T19:16:09.679532+0000","last_became_active":"2024-09-10T19:15:58.835522+0000","last_became_peered":"2024-09-10T19:15:58.835522+0000","last_unstale":"2024-09-10T19:16:09.679532+0000","last_undegraded":"2024-09-10T19:16:09.679532+0000","last_fullsized":"2024-09-10T19:16:09.679532+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:06.306813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679541+0000","last_change":"2024-09-10T19:15:56.814106+0000","last_active":"2024-09-10T19:16:09.679541+0000","last_peered":"2024-09-10T19:16:09.679541+0000","last_clean":"2024-09-10T19:16:09.679541+0000","last_became_active":"2024-09-10T19:15:56.813740+0000","last_became_peered":"2024-09-10T19:15:56.813740+0000","last_unstale":"2024-09-10T19:16:09.679541+0000","last_undegraded":"2024-09-10T19:16:09.679541+0000","last_fullsized":"2024-09-10T19:16:09.679541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:16.778568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309587+0000","last_change":"2024-09-10T19:16:00.828756+0000","last_active":"2024-09-10T19:16:09.309587+0000","last_peered":"2024-09-10T19:16:09.309587+0000","last_clean":"2024-09-10T19:16:09.309587+0000","last_became_active":"2024-09-10T19:16:00.828481+0000","last_became_peered":"2024-09-10T19:16:00.828481+0000","last_unstale":"2024-09-10T19:16:09.309587+0000","last_undegraded":"2024-09-10T19:16:09.309587+0000","last_fullsized":"2024-09-10T19:16:09.309587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:28.472480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309145+0000","last_change":"2024-09-10T19:16:02.843426+0000","last_active":"2024-09-10T19:16:09.309145+0000","last_peered":"2024-09-10T19:16:09.309145+0000","last_clean":"2024-09-10T19:16:09.309145+0000","last_became_active":"2024-09-10T19:16:02.843163+0000","last_became_peered":"2024-09-10T19:16:02.843163+0000","last_unstale":"2024-09-10T19:16:09.309145+0000","last_undegraded":"2024-09-10T19:16:09.309145+0000","last_fullsized":"2024-09-10T19:16:09.309145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:16.365335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308578+0000","last_change":"2024-09-10T19:15:58.831059+0000","last_active":"2024-09-10T19:16:09.308578+0000","last_peered":"2024-09-10T19:16:09.308578+0000","last_clean":"2024-09-10T19:16:09.308578+0000","last_became_active":"2024-09-10T19:15:58.830700+0000","last_became_peered":"2024-09-10T19:15:58.830700+0000","last_unstale":"2024-09-10T19:16:09.308578+0000","last_undegraded":"2024-09-10T19:16:09.308578+0000","last_fullsized":"2024-09-10T19:16:09.308578+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:37.622626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680062+0000","last_change":"2024-09-10T19:15:56.814096+0000","last_active":"2024-09-10T19:16:09.680062+0000","last_peered":"2024-09-10T19:16:09.680062+0000","last_clean":"2024-09-10T19:16:09.680062+0000","last_became_active":"2024-09-10T19:15:56.813796+0000","last_became_peered":"2024-09-10T19:15:56.813796+0000","last_unstale":"2024-09-10T19:16:09.680062+0000","last_undegraded":"2024-09-10T19:16:09.680062+0000","last_fullsized":"2024-09-10T19:16:09.680062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:49.847594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679383+0000","last_change":"2024-09-10T19:16:00.832798+0000","last_active":"2024-09-10T19:16:09.679383+0000","last_peered":"2024-09-10T19:16:09.679383+0000","last_clean":"2024-09-10T19:16:09.679383+0000","last_became_active":"2024-09-10T19:16:00.832514+0000","last_became_peered":"2024-09-10T19:16:00.832514+0000","last_unstale":"2024-09-10T19:16:09.679383+0000","last_undegraded":"2024-09-10T19:16:09.679383+0000","last_fullsized":"2024-09-10T19:16:09.679383+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:13.788951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310043+0000","last_change":"2024-09-10T19:16:02.841281+0000","last_active":"2024-09-10T19:16:09.310043+0000","last_peered":"2024-09-10T19:16:09.310043+0000","last_clean":"2024-09-10T19:16:09.310043+0000","last_became_active":"2024-09-10T19:16:02.840820+0000","last_became_peered":"2024-09-10T19:16:02.840820+0000","last_unstale":"2024-09-10T19:16:09.310043+0000","last_undegraded":"2024-09-10T19:16:09.310043+0000","last_fullsized":"2024-09-10T19:16:09.310043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:19.856229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310255+0000","last_change":"2024-09-10T19:15:58.831283+0000","last_active":"2024-09-10T19:16:09.310255+0000","last_peered":"2024-09-10T19:16:09.310255+0000","last_clean":"2024-09-10T19:16:09.310255+0000","last_became_active":"2024-09-10T19:15:58.830910+0000","last_became_peered":"2024-09-10T19:15:58.830910+0000","last_unstale":"2024-09-10T19:16:09.310255+0000","last_undegraded":"2024-09-10T19:16:09.310255+0000","last_fullsized":"2024-09-10T19:16:09.310255+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:43.005321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680383+0000","last_change":"2024-09-10T19:15:56.812953+0000","last_active":"2024-09-10T19:16:09.680383+0000","last_peered":"2024-09-10T19:16:09.680383+0000","last_clean":"2024-09-10T19:16:09.680383+0000","last_became_active":"2024-09-10T19:15:56.812669+0000","last_became_peered":"2024-09-10T19:15:56.812669+0000","last_unstale":"2024-09-10T19:16:09.680383+0000","last_undegraded":"2024-09-10T19:16:09.680383+0000","last_fullsized":"2024-09-10T19:16:09.680383+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.987254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678763+0000","last_change":"2024-09-10T19:16:00.833086+0000","last_active":"2024-09-10T19:16:09.678763+0000","last_peered":"2024-09-10T19:16:09.678763+0000","last_clean":"2024-09-10T19:16:09.678763+0000","last_became_active":"2024-09-10T19:16:00.832727+0000","last_became_peered":"2024-09-10T19:16:00.832727+0000","last_unstale":"2024-09-10T19:16:09.678763+0000","last_undegraded":"2024-09-10T19:16:09.678763+0000","last_fullsized":"2024-09-10T19:16:09.678763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:03.555485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678827+0000","last_change":"2024-09-10T19:16:02.833685+0000","last_active":"2024-09-10T19:16:09.678827+0000","last_peered":"2024-09-10T19:16:09.678827+0000","last_clean":"2024-09-10T19:16:09.678827+0000","last_became_active":"2024-09-10T19:16:02.833410+0000","last_became_peered":"2024-09-10T19:16:02.833410+0000","last_unstale":"2024-09-10T19:16:09.678827+0000","last_undegraded":"2024-09-10T19:16:09.678827+0000","last_fullsized":"2024-09-10T19:16:09.678827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:44.049592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310366+0000","last_change":"2024-09-10T19:15:59.477484+0000","last_active":"2024-09-10T19:16:09.310366+0000","last_peered":"2024-09-10T19:16:09.310366+0000","last_clean":"2024-09-10T19:16:09.310366+0000","last_became_active":"2024-09-10T19:15:59.477187+0000","last_became_peered":"2024-09-10T19:15:59.477187+0000","last_unstale":"2024-09-10T19:16:09.310366+0000","last_undegraded":"2024-09-10T19:16:09.310366+0000","last_fullsized":"2024-09-10T19:16:09.310366+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:47.343734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680901+0000","last_change":"2024-09-10T19:15:56.813276+0000","last_active":"2024-09-10T19:16:09.680901+0000","last_peered":"2024-09-10T19:16:09.680901+0000","last_clean":"2024-09-10T19:16:09.680901+0000","last_became_active":"2024-09-10T19:15:56.813031+0000","last_became_peered":"2024-09-10T19:15:56.813031+0000","last_unstale":"2024-09-10T19:16:09.680901+0000","last_undegraded":"2024-09-10T19:16:09.680901+0000","last_fullsized":"2024-09-10T19:16:09.680901+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:13.966067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678487+0000","last_change":"2024-09-10T19:16:00.832082+0000","last_active":"2024-09-10T19:16:09.678487+0000","last_peered":"2024-09-10T19:16:09.678487+0000","last_clean":"2024-09-10T19:16:09.678487+0000","last_became_active":"2024-09-10T19:16:00.831524+0000","last_became_peered":"2024-09-10T19:16:00.831524+0000","last_unstale":"2024-09-10T19:16:09.678487+0000","last_undegraded":"2024-09-10T19:16:09.678487+0000","last_fullsized":"2024-09-10T19:16:09.678487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:02.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309425+0000","last_change":"2024-09-10T19:16:03.115843+0000","last_active":"2024-09-10T19:16:09.309425+0000","last_peered":"2024-09-10T19:16:09.309425+0000","last_clean":"2024-09-10T19:16:09.309425+0000","last_became_active":"2024-09-10T19:16:03.115430+0000","last_became_peered":"2024-09-10T19:16:03.115430+0000","last_unstale":"2024-09-10T19:16:09.309425+0000","last_undegraded":"2024-09-10T19:16:09.309425+0000","last_fullsized":"2024-09-10T19:16:09.309425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:05.719257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679465+0000","last_change":"2024-09-10T19:16:02.837432+0000","last_active":"2024-09-10T19:16:09.679465+0000","last_peered":"2024-09-10T19:16:09.679465+0000","last_clean":"2024-09-10T19:16:09.679465+0000","last_became_active":"2024-09-10T19:16:02.837160+0000","last_became_peered":"2024-09-10T19:16:02.837160+0000","last_unstale":"2024-09-10T19:16:09.679465+0000","last_undegraded":"2024-09-10T19:16:09.679465+0000","last_fullsized":"2024-09-10T19:16:09.679465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:58.230781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680777+0000","last_change":"2024-09-10T19:15:56.808289+0000","last_active":"2024-09-10T19:16:09.680777+0000","last_peered":"2024-09-10T19:16:09.680777+0000","last_clean":"2024-09-10T19:16:09.680777+0000","last_became_active":"2024-09-10T19:15:56.807387+0000","last_became_peered":"2024-09-10T19:15:56.807387+0000","last_unstale":"2024-09-10T19:16:09.680777+0000","last_undegraded":"2024-09-10T19:16:09.680777+0000","last_fullsized":"2024-09-10T19:16:09.680777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:48.709035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311107+0000","last_change":"2024-09-10T19:15:59.479758+0000","last_active":"2024-09-10T19:16:09.311107+0000","last_peered":"2024-09-10T19:16:09.311107+0000","last_clean":"2024-09-10T19:16:09.311107+0000","last_became_active":"2024-09-10T19:15:59.479500+0000","last_became_peered":"2024-09-10T19:15:59.479500+0000","last_unstale":"2024-09-10T19:16:09.311107+0000","last_undegraded":"2024-09-10T19:16:09.311107+0000","last_fullsized":"2024-09-10T19:16:09.311107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:48.212860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167460+0000","last_change":"2024-09-10T19:16:00.835173+0000","last_active":"2024-09-10T19:17:10.167460+0000","last_peered":"2024-09-10T19:17:10.167460+0000","last_clean":"2024-09-10T19:17:10.167460+0000","last_became_active":"2024-09-10T19:16:00.834417+0000","last_became_peered":"2024-09-10T19:16:00.834417+0000","last_unstale":"2024-09-10T19:17:10.167460+0000","last_undegraded":"2024-09-10T19:17:10.167460+0000","last_fullsized":"2024-09-10T19:17:10.167460+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:36.906340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679756+0000","last_change":"2024-09-10T19:16:02.842552+0000","last_active":"2024-09-10T19:16:09.679756+0000","last_peered":"2024-09-10T19:16:09.679756+0000","last_clean":"2024-09-10T19:16:09.679756+0000","last_became_active":"2024-09-10T19:16:02.842238+0000","last_became_peered":"2024-09-10T19:16:02.842238+0000","last_unstale":"2024-09-10T19:16:09.679756+0000","last_undegraded":"2024-09-10T19:16:09.679756+0000","last_fullsized":"2024-09-10T19:16:09.679756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:54.648225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310635+0000","last_change":"2024-09-10T19:15:56.811746+0000","last_active":"2024-09-10T19:16:09.310635+0000","last_peered":"2024-09-10T19:16:09.310635+0000","last_clean":"2024-09-10T19:16:09.310635+0000","last_became_active":"2024-09-10T19:15:56.811503+0000","last_became_peered":"2024-09-10T19:15:56.811503+0000","last_unstale":"2024-09-10T19:16:09.310635+0000","last_undegraded":"2024-09-10T19:16:09.310635+0000","last_fullsized":"2024-09-10T19:16:09.310635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:33.995220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680823+0000","last_change":"2024-09-10T19:15:58.830917+0000","last_active":"2024-09-10T19:16:09.680823+0000","last_peered":"2024-09-10T19:16:09.680823+0000","last_clean":"2024-09-10T19:16:09.680823+0000","last_became_active":"2024-09-10T19:15:58.830623+0000","last_became_peered":"2024-09-10T19:15:58.830623+0000","last_unstale":"2024-09-10T19:16:09.680823+0000","last_undegraded":"2024-09-10T19:16:09.680823+0000","last_fullsized":"2024-09-10T19:16:09.680823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:32.904029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311010+0000","last_change":"2024-09-10T19:16:00.835141+0000","last_active":"2024-09-10T19:16:09.311010+0000","last_peered":"2024-09-10T19:16:09.311010+0000","last_clean":"2024-09-10T19:16:09.311010+0000","last_became_active":"2024-09-10T19:16:00.834727+0000","last_became_peered":"2024-09-10T19:16:00.834727+0000","last_unstale":"2024-09-10T19:16:09.311010+0000","last_undegraded":"2024-09-10T19:16:09.311010+0000","last_fullsized":"2024-09-10T19:16:09.311010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:52.987521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308691+0000","last_change":"2024-09-10T19:16:03.115482+0000","last_active":"2024-09-10T19:16:09.308691+0000","last_peered":"2024-09-10T19:16:09.308691+0000","last_clean":"2024-09-10T19:16:09.308691+0000","last_became_active":"2024-09-10T19:16:03.115201+0000","last_became_peered":"2024-09-10T19:16:03.115201+0000","last_unstale":"2024-09-10T19:16:09.308691+0000","last_undegraded":"2024-09-10T19:16:09.308691+0000","last_fullsized":"2024-09-10T19:16:09.308691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:34.609845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681000+0000","last_change":"2024-09-10T19:15:56.810432+0000","last_active":"2024-09-10T19:16:09.681000+0000","last_peered":"2024-09-10T19:16:09.681000+0000","last_clean":"2024-09-10T19:16:09.681000+0000","last_became_active":"2024-09-10T19:15:56.808882+0000","last_became_peered":"2024-09-10T19:15:56.808882+0000","last_unstale":"2024-09-10T19:16:09.681000+0000","last_undegraded":"2024-09-10T19:16:09.681000+0000","last_fullsized":"2024-09-10T19:16:09.681000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:17.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679036+0000","last_change":"2024-09-10T19:15:58.832344+0000","last_active":"2024-09-10T19:16:09.679036+0000","last_peered":"2024-09-10T19:16:09.679036+0000","last_clean":"2024-09-10T19:16:09.679036+0000","last_became_active":"2024-09-10T19:15:58.832022+0000","last_became_peered":"2024-09-10T19:15:58.832022+0000","last_unstale":"2024-09-10T19:16:09.679036+0000","last_undegraded":"2024-09-10T19:16:09.679036+0000","last_fullsized":"2024-09-10T19:16:09.679036+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:17.309755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679949+0000","last_change":"2024-09-10T19:16:00.833127+0000","last_active":"2024-09-10T19:16:09.679949+0000","last_peered":"2024-09-10T19:16:09.679949+0000","last_clean":"2024-09-10T19:16:09.679949+0000","last_became_active":"2024-09-10T19:16:00.832866+0000","last_became_peered":"2024-09-10T19:16:00.832866+0000","last_unstale":"2024-09-10T19:16:09.679949+0000","last_undegraded":"2024-09-10T19:16:09.679949+0000","last_fullsized":"2024-09-10T19:16:09.679949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:36.173793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310929+0000","last_change":"2024-09-10T19:16:03.115614+0000","last_active":"2024-09-10T19:16:09.310929+0000","last_peered":"2024-09-10T19:16:09.310929+0000","last_clean":"2024-09-10T19:16:09.310929+0000","last_became_active":"2024-09-10T19:16:03.115024+0000","last_became_peered":"2024-09-10T19:16:03.115024+0000","last_unstale":"2024-09-10T19:16:09.310929+0000","last_undegraded":"2024-09-10T19:16:09.310929+0000","last_fullsized":"2024-09-10T19:16:09.310929+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:36.701997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309799+0000","last_change":"2024-09-10T19:15:56.810022+0000","last_active":"2024-09-10T19:16:09.309799+0000","last_peered":"2024-09-10T19:16:09.309799+0000","last_clean":"2024-09-10T19:16:09.309799+0000","last_became_active":"2024-09-10T19:15:56.809776+0000","last_became_peered":"2024-09-10T19:15:56.809776+0000","last_unstale":"2024-09-10T19:16:09.309799+0000","last_undegraded":"2024-09-10T19:16:09.309799+0000","last_fullsized":"2024-09-10T19:16:09.309799+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:06.855548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309486+0000","last_change":"2024-09-10T19:15:58.830445+0000","last_active":"2024-09-10T19:16:09.309486+0000","last_peered":"2024-09-10T19:16:09.309486+0000","last_clean":"2024-09-10T19:16:09.309486+0000","last_became_active":"2024-09-10T19:15:58.830053+0000","last_became_peered":"2024-09-10T19:15:58.830053+0000","last_unstale":"2024-09-10T19:16:09.309486+0000","last_undegraded":"2024-09-10T19:16:09.309486+0000","last_fullsized":"2024-09-10T19:16:09.309486+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:51.165963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308636+0000","last_change":"2024-09-10T19:16:00.833524+0000","last_active":"2024-09-10T19:16:09.308636+0000","last_peered":"2024-09-10T19:16:09.308636+0000","last_clean":"2024-09-10T19:16:09.308636+0000","last_became_active":"2024-09-10T19:16:00.833210+0000","last_became_peered":"2024-09-10T19:16:00.833210+0000","last_unstale":"2024-09-10T19:16:09.308636+0000","last_undegraded":"2024-09-10T19:16:09.308636+0000","last_fullsized":"2024-09-10T19:16:09.308636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:12.600185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397525,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1216512,"data_stored":825817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.503},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.35899999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40100000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1204224,"data_stored":825544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499}]},{"osd":1,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629}]},{"osd":3,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.36699999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.438}]}]},{"osd":5,"up_from":37,"seq":158913789983,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":0.69399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":18.146999999999998,"5min":18.146999999999998,"15min":18.146999999999998},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67100000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.29299999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.505}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.372},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33600000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40300000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":234439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58699999999999997}]},{"osd":1,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60099999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.753,"15min":0.753},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.373,"15min":0.36499999999999999},"min":{"1min":0.23699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42299999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":235139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.35499999999999998,"15min":0.36399999999999999},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40899999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.375,"15min":0.379},"min":{"1min":0.26400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38600000000000001,"15min":0.39400000000000002},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.373,"15min":0.38100000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.498,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.315},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.52500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.26500000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.38700000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.745,"5min":1.3,"15min":1.3},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":234219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.221,"5min":0.214,"15min":0.214},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.35999999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.373}]},{"osd":1,"last update":"Tue Sep 10 19:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.377,"15min":0.373},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38700000000000001,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.29499999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45500000000000002,"15min":0.46100000000000002},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.65000000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.308}]},{"osd":5,"last update":"Tue Sep 10 19:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.33900000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.443},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34699999999999998}]},{"osd":7,"last update":"Tue Sep 10 19:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.34300000000000003},"min":{"1min":0.23799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.374},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.374},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51000000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.433},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":825876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53500000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.249},{"interface":"front","average":{"1min":0.378,"5min":0.34599999999999997,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.378},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45800000000000002,"5min":0.63,"15min":0.63},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.47699999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.314}]},{"osd":3,"last update":"Tue Sep 10 19:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.41699999999999998,"15min":0.436},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.441,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.505,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.28199999999999997}]},{"osd":4,"last update":"Tue Sep 10 19:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70699999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.84599999999999997,"15min":0.88800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":6.9509999999999996,"15min":6.9509999999999996},"last":0.375}]},{"osd":6,"last update":"Tue Sep 10 19:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.976,"5min":6.976,"15min":6.976},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":12.502000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:17:20.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T19:17:21.133 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:21.411 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:21 smithi115 ceph-mon[28968]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:21.411 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:21 smithi115 ceph-mon[39520]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:21.514 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:21 smithi150 ceph-mon[33096]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:22.227 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:22 smithi115 ceph-mon[39520]: from='client.14730 v1:172.21.15.115:0/544424756' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:22.227 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:22 smithi115 ceph-mon[28968]: from='client.14730 v1:172.21.15.115:0/544424756' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:22.586 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:22 smithi150 ceph-mon[33096]: from='client.14730 v1:172.21.15.115:0/544424756' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:22.900 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:17:22.902 INFO:teuthology.orchestra.run.smithi115.stderr:dumped all 2024-09-10T19:17:23.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:23 smithi150 ceph-mon[33096]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:23 smithi115 ceph-mon[28968]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:23.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:23 smithi115 ceph-mon[39520]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:23.491 INFO:teuthology.orchestra.run.smithi115.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T19:17:21.709173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":928,"num_read_kb":785,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385456,"kb_used_data":6612,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575760,"statfs":{"total":767960285184,"available":765517578240,"internally_reserved":0,"allocated":6770688,"data_stored":3650326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002604"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309346+0000","last_change":"2024-09-10T19:16:03.116640+0000","last_active":"2024-09-10T19:16:09.309346+0000","last_peered":"2024-09-10T19:16:09.309346+0000","last_clean":"2024-09-10T19:16:09.309346+0000","last_became_active":"2024-09-10T19:16:03.116303+0000","last_became_peered":"2024-09-10T19:16:03.116303+0000","last_unstale":"2024-09-10T19:16:09.309346+0000","last_undegraded":"2024-09-10T19:16:09.309346+0000","last_fullsized":"2024-09-10T19:16:09.309346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:23.329001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310754+0000","last_change":"2024-09-10T19:15:56.812467+0000","last_active":"2024-09-10T19:16:09.310754+0000","last_peered":"2024-09-10T19:16:09.310754+0000","last_clean":"2024-09-10T19:16:09.310754+0000","last_became_active":"2024-09-10T19:15:56.811971+0000","last_became_peered":"2024-09-10T19:15:56.811971+0000","last_unstale":"2024-09-10T19:16:09.310754+0000","last_undegraded":"2024-09-10T19:16:09.310754+0000","last_fullsized":"2024-09-10T19:16:09.310754+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:37.539871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310473+0000","last_change":"2024-09-10T19:15:58.824733+0000","last_active":"2024-09-10T19:16:09.310473+0000","last_peered":"2024-09-10T19:16:09.310473+0000","last_clean":"2024-09-10T19:16:09.310473+0000","last_became_active":"2024-09-10T19:15:58.824400+0000","last_became_peered":"2024-09-10T19:15:58.824400+0000","last_unstale":"2024-09-10T19:16:09.310473+0000","last_undegraded":"2024-09-10T19:16:09.310473+0000","last_fullsized":"2024-09-10T19:16:09.310473+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:53.747455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308265+0000","last_change":"2024-09-10T19:16:00.839328+0000","last_active":"2024-09-10T19:16:09.308265+0000","last_peered":"2024-09-10T19:16:09.308265+0000","last_clean":"2024-09-10T19:16:09.308265+0000","last_became_active":"2024-09-10T19:16:00.838755+0000","last_became_peered":"2024-09-10T19:16:00.838755+0000","last_unstale":"2024-09-10T19:16:09.308265+0000","last_undegraded":"2024-09-10T19:16:09.308265+0000","last_fullsized":"2024-09-10T19:16:09.308265+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:04.387113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310929+0000","last_change":"2024-09-10T19:15:56.809353+0000","last_active":"2024-09-10T19:16:09.310929+0000","last_peered":"2024-09-10T19:16:09.310929+0000","last_clean":"2024-09-10T19:16:09.310929+0000","last_became_active":"2024-09-10T19:15:56.806738+0000","last_became_peered":"2024-09-10T19:15:56.806738+0000","last_unstale":"2024-09-10T19:16:09.310929+0000","last_undegraded":"2024-09-10T19:16:09.310929+0000","last_fullsized":"2024-09-10T19:16:09.310929+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:48.924228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308273+0000","last_change":"2024-09-10T19:15:58.832461+0000","last_active":"2024-09-10T19:16:09.308273+0000","last_peered":"2024-09-10T19:16:09.308273+0000","last_clean":"2024-09-10T19:16:09.308273+0000","last_became_active":"2024-09-10T19:15:58.832157+0000","last_became_peered":"2024-09-10T19:15:58.832157+0000","last_unstale":"2024-09-10T19:16:09.308273+0000","last_undegraded":"2024-09-10T19:16:09.308273+0000","last_fullsized":"2024-09-10T19:16:09.308273+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:36:06.592665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679004+0000","last_change":"2024-09-10T19:16:00.831165+0000","last_active":"2024-09-10T19:16:09.679004+0000","last_peered":"2024-09-10T19:16:09.679004+0000","last_clean":"2024-09-10T19:16:09.679004+0000","last_became_active":"2024-09-10T19:16:00.830900+0000","last_became_peered":"2024-09-10T19:16:00.830900+0000","last_unstale":"2024-09-10T19:16:09.679004+0000","last_undegraded":"2024-09-10T19:16:09.679004+0000","last_fullsized":"2024-09-10T19:16:09.679004+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:34.289163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308314+0000","last_change":"2024-09-10T19:16:02.841379+0000","last_active":"2024-09-10T19:16:09.308314+0000","last_peered":"2024-09-10T19:16:09.308314+0000","last_clean":"2024-09-10T19:16:09.308314+0000","last_became_active":"2024-09-10T19:16:02.841132+0000","last_became_peered":"2024-09-10T19:16:02.841132+0000","last_unstale":"2024-09-10T19:16:09.308314+0000","last_undegraded":"2024-09-10T19:16:09.308314+0000","last_fullsized":"2024-09-10T19:16:09.308314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:16:32.573021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"53'1","reported_seq":27,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680045+0000","last_change":"2024-09-10T19:15:56.811386+0000","last_active":"2024-09-10T19:16:09.680045+0000","last_peered":"2024-09-10T19:16:09.680045+0000","last_clean":"2024-09-10T19:16:09.680045+0000","last_became_active":"2024-09-10T19:15:56.811107+0000","last_became_peered":"2024-09-10T19:15:56.811107+0000","last_unstale":"2024-09-10T19:16:09.680045+0000","last_undegraded":"2024-09-10T19:16:09.680045+0000","last_fullsized":"2024-09-10T19:16:09.680045+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:34.798637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"60'5","reported_seq":26,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308761+0000","last_change":"2024-09-10T19:15:58.831860+0000","last_active":"2024-09-10T19:16:09.308761+0000","last_peered":"2024-09-10T19:16:09.308761+0000","last_clean":"2024-09-10T19:16:09.308761+0000","last_became_active":"2024-09-10T19:15:58.831504+0000","last_became_peered":"2024-09-10T19:15:58.831504+0000","last_unstale":"2024-09-10T19:16:09.308761+0000","last_undegraded":"2024-09-10T19:16:09.308761+0000","last_fullsized":"2024-09-10T19:16:09.308761+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:49.655790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678894+0000","last_change":"2024-09-10T19:16:00.832542+0000","last_active":"2024-09-10T19:16:09.678894+0000","last_peered":"2024-09-10T19:16:09.678894+0000","last_clean":"2024-09-10T19:16:09.678894+0000","last_became_active":"2024-09-10T19:16:00.832217+0000","last_became_peered":"2024-09-10T19:16:00.832217+0000","last_unstale":"2024-09-10T19:16:09.678894+0000","last_undegraded":"2024-09-10T19:16:09.678894+0000","last_fullsized":"2024-09-10T19:16:09.678894+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:37.198765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678671+0000","last_change":"2024-09-10T19:16:02.840530+0000","last_active":"2024-09-10T19:16:09.678671+0000","last_peered":"2024-09-10T19:16:09.678671+0000","last_clean":"2024-09-10T19:16:09.678671+0000","last_became_active":"2024-09-10T19:16:02.840217+0000","last_became_peered":"2024-09-10T19:16:02.840217+0000","last_unstale":"2024-09-10T19:16:09.678671+0000","last_undegraded":"2024-09-10T19:16:09.678671+0000","last_fullsized":"2024-09-10T19:16:09.678671+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:08.780416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679793+0000","last_change":"2024-09-10T19:15:56.807970+0000","last_active":"2024-09-10T19:16:09.679793+0000","last_peered":"2024-09-10T19:16:09.679793+0000","last_clean":"2024-09-10T19:16:09.679793+0000","last_became_active":"2024-09-10T19:15:56.807652+0000","last_became_peered":"2024-09-10T19:15:56.807652+0000","last_unstale":"2024-09-10T19:16:09.679793+0000","last_undegraded":"2024-09-10T19:16:09.679793+0000","last_fullsized":"2024-09-10T19:16:09.679793+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:23.013672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308755+0000","last_change":"2024-09-10T19:15:58.823033+0000","last_active":"2024-09-10T19:16:09.308755+0000","last_peered":"2024-09-10T19:16:09.308755+0000","last_clean":"2024-09-10T19:16:09.308755+0000","last_became_active":"2024-09-10T19:15:58.822704+0000","last_became_peered":"2024-09-10T19:15:58.822704+0000","last_unstale":"2024-09-10T19:16:09.308755+0000","last_undegraded":"2024-09-10T19:16:09.308755+0000","last_fullsized":"2024-09-10T19:16:09.308755+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:20.892062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678701+0000","last_change":"2024-09-10T19:16:00.831882+0000","last_active":"2024-09-10T19:16:09.678701+0000","last_peered":"2024-09-10T19:16:09.678701+0000","last_clean":"2024-09-10T19:16:09.678701+0000","last_became_active":"2024-09-10T19:16:00.831604+0000","last_became_peered":"2024-09-10T19:16:00.831604+0000","last_unstale":"2024-09-10T19:16:09.678701+0000","last_undegraded":"2024-09-10T19:16:09.678701+0000","last_fullsized":"2024-09-10T19:16:09.678701+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:23:09.222274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309301+0000","last_change":"2024-09-10T19:16:02.843903+0000","last_active":"2024-09-10T19:16:09.309301+0000","last_peered":"2024-09-10T19:16:09.309301+0000","last_clean":"2024-09-10T19:16:09.309301+0000","last_became_active":"2024-09-10T19:16:02.843631+0000","last_became_peered":"2024-09-10T19:16:02.843631+0000","last_unstale":"2024-09-10T19:16:09.309301+0000","last_undegraded":"2024-09-10T19:16:09.309301+0000","last_fullsized":"2024-09-10T19:16:09.309301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:44:22.956435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"60'17","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308933+0000","last_change":"2024-09-10T19:15:58.827148+0000","last_active":"2024-09-10T19:16:09.308933+0000","last_peered":"2024-09-10T19:16:09.308933+0000","last_clean":"2024-09-10T19:16:09.308933+0000","last_became_active":"2024-09-10T19:15:58.826812+0000","last_became_peered":"2024-09-10T19:15:58.826812+0000","last_unstale":"2024-09-10T19:16:09.308933+0000","last_undegraded":"2024-09-10T19:16:09.308933+0000","last_fullsized":"2024-09-10T19:16:09.308933+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:54.915553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679578+0000","last_change":"2024-09-10T19:15:56.807618+0000","last_active":"2024-09-10T19:16:09.679578+0000","last_peered":"2024-09-10T19:16:09.679578+0000","last_clean":"2024-09-10T19:16:09.679578+0000","last_became_active":"2024-09-10T19:15:56.807218+0000","last_became_peered":"2024-09-10T19:15:56.807218+0000","last_unstale":"2024-09-10T19:16:09.679578+0000","last_undegraded":"2024-09-10T19:16:09.679578+0000","last_fullsized":"2024-09-10T19:16:09.679578+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:46:21.356289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679640+0000","last_change":"2024-09-10T19:16:00.832475+0000","last_active":"2024-09-10T19:16:09.679640+0000","last_peered":"2024-09-10T19:16:09.679640+0000","last_clean":"2024-09-10T19:16:09.679640+0000","last_became_active":"2024-09-10T19:16:00.831978+0000","last_became_peered":"2024-09-10T19:16:00.831978+0000","last_unstale":"2024-09-10T19:16:09.679640+0000","last_undegraded":"2024-09-10T19:16:09.679640+0000","last_fullsized":"2024-09-10T19:16:09.679640+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:01.624513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678887+0000","last_change":"2024-09-10T19:16:02.837393+0000","last_active":"2024-09-10T19:16:09.678887+0000","last_peered":"2024-09-10T19:16:09.678887+0000","last_clean":"2024-09-10T19:16:09.678887+0000","last_became_active":"2024-09-10T19:16:02.833233+0000","last_became_peered":"2024-09-10T19:16:02.833233+0000","last_unstale":"2024-09-10T19:16:09.678887+0000","last_undegraded":"2024-09-10T19:16:09.678887+0000","last_fullsized":"2024-09-10T19:16:09.678887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:03.182381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308591+0000","last_change":"2024-09-10T19:15:58.824660+0000","last_active":"2024-09-10T19:16:09.308591+0000","last_peered":"2024-09-10T19:16:09.308591+0000","last_clean":"2024-09-10T19:16:09.308591+0000","last_became_active":"2024-09-10T19:15:58.824328+0000","last_became_peered":"2024-09-10T19:15:58.824328+0000","last_unstale":"2024-09-10T19:16:09.308591+0000","last_undegraded":"2024-09-10T19:16:09.308591+0000","last_fullsized":"2024-09-10T19:16:09.308591+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:00.531536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679672+0000","last_change":"2024-09-10T19:15:56.818384+0000","last_active":"2024-09-10T19:16:09.679672+0000","last_peered":"2024-09-10T19:16:09.679672+0000","last_clean":"2024-09-10T19:16:09.679672+0000","last_became_active":"2024-09-10T19:15:56.817911+0000","last_became_peered":"2024-09-10T19:15:56.817911+0000","last_unstale":"2024-09-10T19:16:09.679672+0000","last_undegraded":"2024-09-10T19:16:09.679672+0000","last_fullsized":"2024-09-10T19:16:09.679672+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:53.905712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166971+0000","last_change":"2024-09-10T19:16:00.834040+0000","last_active":"2024-09-10T19:17:10.166971+0000","last_peered":"2024-09-10T19:17:10.166971+0000","last_clean":"2024-09-10T19:17:10.166971+0000","last_became_active":"2024-09-10T19:16:00.833736+0000","last_became_peered":"2024-09-10T19:16:00.833736+0000","last_unstale":"2024-09-10T19:17:10.166971+0000","last_undegraded":"2024-09-10T19:17:10.166971+0000","last_fullsized":"2024-09-10T19:17:10.166971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:23.714061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308664+0000","last_change":"2024-09-10T19:16:02.833914+0000","last_active":"2024-09-10T19:16:09.308664+0000","last_peered":"2024-09-10T19:16:09.308664+0000","last_clean":"2024-09-10T19:16:09.308664+0000","last_became_active":"2024-09-10T19:16:02.833627+0000","last_became_peered":"2024-09-10T19:16:02.833627+0000","last_unstale":"2024-09-10T19:16:09.308664+0000","last_undegraded":"2024-09-10T19:16:09.308664+0000","last_fullsized":"2024-09-10T19:16:09.308664+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:48.704666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680104+0000","last_change":"2024-09-10T19:15:58.831933+0000","last_active":"2024-09-10T19:16:09.680104+0000","last_peered":"2024-09-10T19:16:09.680104+0000","last_clean":"2024-09-10T19:16:09.680104+0000","last_became_active":"2024-09-10T19:15:58.831679+0000","last_became_peered":"2024-09-10T19:15:58.831679+0000","last_unstale":"2024-09-10T19:16:09.680104+0000","last_undegraded":"2024-09-10T19:16:09.680104+0000","last_fullsized":"2024-09-10T19:16:09.680104+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:34.650482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679301+0000","last_change":"2024-09-10T19:15:56.820108+0000","last_active":"2024-09-10T19:16:09.679301+0000","last_peered":"2024-09-10T19:16:09.679301+0000","last_clean":"2024-09-10T19:16:09.679301+0000","last_became_active":"2024-09-10T19:15:56.818080+0000","last_became_peered":"2024-09-10T19:15:56.818080+0000","last_unstale":"2024-09-10T19:16:09.679301+0000","last_undegraded":"2024-09-10T19:16:09.679301+0000","last_fullsized":"2024-09-10T19:16:09.679301+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:01:57.762435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167244+0000","last_change":"2024-09-10T19:16:00.836710+0000","last_active":"2024-09-10T19:17:10.167244+0000","last_peered":"2024-09-10T19:17:10.167244+0000","last_clean":"2024-09-10T19:17:10.167244+0000","last_became_active":"2024-09-10T19:16:00.836448+0000","last_became_peered":"2024-09-10T19:16:00.836448+0000","last_unstale":"2024-09-10T19:17:10.167244+0000","last_undegraded":"2024-09-10T19:17:10.167244+0000","last_fullsized":"2024-09-10T19:17:10.167244+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:39.379233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680965+0000","last_change":"2024-09-10T19:16:02.840540+0000","last_active":"2024-09-10T19:16:09.680965+0000","last_peered":"2024-09-10T19:16:09.680965+0000","last_clean":"2024-09-10T19:16:09.680965+0000","last_became_active":"2024-09-10T19:16:02.840269+0000","last_became_peered":"2024-09-10T19:16:02.840269+0000","last_unstale":"2024-09-10T19:16:09.680965+0000","last_undegraded":"2024-09-10T19:16:09.680965+0000","last_fullsized":"2024-09-10T19:16:09.680965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:20.222712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308792+0000","last_change":"2024-09-10T19:15:58.827176+0000","last_active":"2024-09-10T19:16:09.308792+0000","last_peered":"2024-09-10T19:16:09.308792+0000","last_clean":"2024-09-10T19:16:09.308792+0000","last_became_active":"2024-09-10T19:15:58.826811+0000","last_became_peered":"2024-09-10T19:15:58.826811+0000","last_unstale":"2024-09-10T19:16:09.308792+0000","last_undegraded":"2024-09-10T19:16:09.308792+0000","last_fullsized":"2024-09-10T19:16:09.308792+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:13.938654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679971+0000","last_change":"2024-09-10T19:15:56.812118+0000","last_active":"2024-09-10T19:16:09.679971+0000","last_peered":"2024-09-10T19:16:09.679971+0000","last_clean":"2024-09-10T19:16:09.679971+0000","last_became_active":"2024-09-10T19:15:56.811855+0000","last_became_peered":"2024-09-10T19:15:56.811855+0000","last_unstale":"2024-09-10T19:16:09.679971+0000","last_undegraded":"2024-09-10T19:16:09.679971+0000","last_fullsized":"2024-09-10T19:16:09.679971+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:22.773134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680204+0000","last_change":"2024-09-10T19:16:00.833046+0000","last_active":"2024-09-10T19:16:09.680204+0000","last_peered":"2024-09-10T19:16:09.680204+0000","last_clean":"2024-09-10T19:16:09.680204+0000","last_became_active":"2024-09-10T19:16:00.832643+0000","last_became_peered":"2024-09-10T19:16:00.832643+0000","last_unstale":"2024-09-10T19:16:09.680204+0000","last_undegraded":"2024-09-10T19:16:09.680204+0000","last_fullsized":"2024-09-10T19:16:09.680204+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:07.615118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310914+0000","last_change":"2024-09-10T19:16:03.115413+0000","last_active":"2024-09-10T19:16:09.310914+0000","last_peered":"2024-09-10T19:16:09.310914+0000","last_clean":"2024-09-10T19:16:09.310914+0000","last_became_active":"2024-09-10T19:16:03.115049+0000","last_became_peered":"2024-09-10T19:16:03.115049+0000","last_unstale":"2024-09-10T19:16:09.310914+0000","last_undegraded":"2024-09-10T19:16:09.310914+0000","last_fullsized":"2024-09-10T19:16:09.310914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:02.490186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"60'14","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308384+0000","last_change":"2024-09-10T19:15:58.824855+0000","last_active":"2024-09-10T19:16:09.308384+0000","last_peered":"2024-09-10T19:16:09.308384+0000","last_clean":"2024-09-10T19:16:09.308384+0000","last_became_active":"2024-09-10T19:15:58.824549+0000","last_became_peered":"2024-09-10T19:15:58.824549+0000","last_unstale":"2024-09-10T19:16:09.308384+0000","last_undegraded":"2024-09-10T19:16:09.308384+0000","last_fullsized":"2024-09-10T19:16:09.308384+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:13.961240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681064+0000","last_change":"2024-09-10T19:15:56.809850+0000","last_active":"2024-09-10T19:16:09.681064+0000","last_peered":"2024-09-10T19:16:09.681064+0000","last_clean":"2024-09-10T19:16:09.681064+0000","last_became_active":"2024-09-10T19:15:56.809450+0000","last_became_peered":"2024-09-10T19:15:56.809450+0000","last_unstale":"2024-09-10T19:16:09.681064+0000","last_undegraded":"2024-09-10T19:16:09.681064+0000","last_fullsized":"2024-09-10T19:16:09.681064+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:56:10.583399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166970+0000","last_change":"2024-09-10T19:16:00.826982+0000","last_active":"2024-09-10T19:17:10.166970+0000","last_peered":"2024-09-10T19:17:10.166970+0000","last_clean":"2024-09-10T19:17:10.166970+0000","last_became_active":"2024-09-10T19:16:00.826661+0000","last_became_peered":"2024-09-10T19:16:00.826661+0000","last_unstale":"2024-09-10T19:17:10.166970+0000","last_undegraded":"2024-09-10T19:17:10.166970+0000","last_fullsized":"2024-09-10T19:17:10.166970+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:42.772632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679835+0000","last_change":"2024-09-10T19:16:03.116078+0000","last_active":"2024-09-10T19:16:09.679835+0000","last_peered":"2024-09-10T19:16:09.679835+0000","last_clean":"2024-09-10T19:16:09.679835+0000","last_became_active":"2024-09-10T19:16:03.115796+0000","last_became_peered":"2024-09-10T19:16:03.115796+0000","last_unstale":"2024-09-10T19:16:09.679835+0000","last_undegraded":"2024-09-10T19:16:09.679835+0000","last_fullsized":"2024-09-10T19:16:09.679835+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:09.875859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"60'18","reported_seq":48,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310360+0000","last_change":"2024-09-10T19:15:59.480620+0000","last_active":"2024-09-10T19:16:09.310360+0000","last_peered":"2024-09-10T19:16:09.310360+0000","last_clean":"2024-09-10T19:16:09.310360+0000","last_became_active":"2024-09-10T19:15:59.480158+0000","last_became_peered":"2024-09-10T19:15:59.480158+0000","last_unstale":"2024-09-10T19:16:09.310360+0000","last_undegraded":"2024-09-10T19:16:09.310360+0000","last_fullsized":"2024-09-10T19:16:09.310360+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:18.312669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680048+0000","last_change":"2024-09-10T19:15:56.816185+0000","last_active":"2024-09-10T19:16:09.680048+0000","last_peered":"2024-09-10T19:16:09.680048+0000","last_clean":"2024-09-10T19:16:09.680048+0000","last_became_active":"2024-09-10T19:15:56.815539+0000","last_became_peered":"2024-09-10T19:15:56.815539+0000","last_unstale":"2024-09-10T19:16:09.680048+0000","last_undegraded":"2024-09-10T19:16:09.680048+0000","last_fullsized":"2024-09-10T19:16:09.680048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:01.145410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308300+0000","last_change":"2024-09-10T19:16:00.838585+0000","last_active":"2024-09-10T19:16:09.308300+0000","last_peered":"2024-09-10T19:16:09.308300+0000","last_clean":"2024-09-10T19:16:09.308300+0000","last_became_active":"2024-09-10T19:16:00.838141+0000","last_became_peered":"2024-09-10T19:16:00.838141+0000","last_unstale":"2024-09-10T19:16:09.308300+0000","last_undegraded":"2024-09-10T19:16:09.308300+0000","last_fullsized":"2024-09-10T19:16:09.308300+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:31.366436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308676+0000","last_change":"2024-09-10T19:16:02.831571+0000","last_active":"2024-09-10T19:16:09.308676+0000","last_peered":"2024-09-10T19:16:09.308676+0000","last_clean":"2024-09-10T19:16:09.308676+0000","last_became_active":"2024-09-10T19:16:02.831267+0000","last_became_peered":"2024-09-10T19:16:02.831267+0000","last_unstale":"2024-09-10T19:16:09.308676+0000","last_undegraded":"2024-09-10T19:16:09.308676+0000","last_fullsized":"2024-09-10T19:16:09.308676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:43.395500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"60'19","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311422+0000","last_change":"2024-09-10T19:15:59.477546+0000","last_active":"2024-09-10T19:16:09.311422+0000","last_peered":"2024-09-10T19:16:09.311422+0000","last_clean":"2024-09-10T19:16:09.311422+0000","last_became_active":"2024-09-10T19:15:59.477251+0000","last_became_peered":"2024-09-10T19:15:59.477251+0000","last_unstale":"2024-09-10T19:16:09.311422+0000","last_undegraded":"2024-09-10T19:16:09.311422+0000","last_fullsized":"2024-09-10T19:16:09.311422+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:11:52.122774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680114+0000","last_change":"2024-09-10T19:15:56.808503+0000","last_active":"2024-09-10T19:16:09.680114+0000","last_peered":"2024-09-10T19:16:09.680114+0000","last_clean":"2024-09-10T19:16:09.680114+0000","last_became_active":"2024-09-10T19:15:56.808237+0000","last_became_peered":"2024-09-10T19:15:56.808237+0000","last_unstale":"2024-09-10T19:16:09.680114+0000","last_undegraded":"2024-09-10T19:16:09.680114+0000","last_fullsized":"2024-09-10T19:16:09.680114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:46:08.346896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680692+0000","last_change":"2024-09-10T19:16:00.838984+0000","last_active":"2024-09-10T19:16:09.680692+0000","last_peered":"2024-09-10T19:16:09.680692+0000","last_clean":"2024-09-10T19:16:09.680692+0000","last_became_active":"2024-09-10T19:16:00.837939+0000","last_became_peered":"2024-09-10T19:16:00.837939+0000","last_unstale":"2024-09-10T19:16:09.680692+0000","last_undegraded":"2024-09-10T19:16:09.680692+0000","last_fullsized":"2024-09-10T19:16:09.680692+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:34.842017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678642+0000","last_change":"2024-09-10T19:16:03.115446+0000","last_active":"2024-09-10T19:16:09.678642+0000","last_peered":"2024-09-10T19:16:09.678642+0000","last_clean":"2024-09-10T19:16:09.678642+0000","last_became_active":"2024-09-10T19:16:03.114988+0000","last_became_peered":"2024-09-10T19:16:03.114988+0000","last_unstale":"2024-09-10T19:16:09.678642+0000","last_undegraded":"2024-09-10T19:16:09.678642+0000","last_fullsized":"2024-09-10T19:16:09.678642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:43:16.052478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679550+0000","last_change":"2024-09-10T19:15:58.828850+0000","last_active":"2024-09-10T19:16:09.679550+0000","last_peered":"2024-09-10T19:16:09.679550+0000","last_clean":"2024-09-10T19:16:09.679550+0000","last_became_active":"2024-09-10T19:15:58.828476+0000","last_became_peered":"2024-09-10T19:15:58.828476+0000","last_unstale":"2024-09-10T19:16:09.679550+0000","last_undegraded":"2024-09-10T19:16:09.679550+0000","last_fullsized":"2024-09-10T19:16:09.679550+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:10.357022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679797+0000","last_change":"2024-09-10T19:15:56.816109+0000","last_active":"2024-09-10T19:16:09.679797+0000","last_peered":"2024-09-10T19:16:09.679797+0000","last_clean":"2024-09-10T19:16:09.679797+0000","last_became_active":"2024-09-10T19:15:56.815830+0000","last_became_peered":"2024-09-10T19:15:56.815830+0000","last_unstale":"2024-09-10T19:16:09.679797+0000","last_undegraded":"2024-09-10T19:16:09.679797+0000","last_fullsized":"2024-09-10T19:16:09.679797+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:40.353434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166971+0000","last_change":"2024-09-10T19:16:00.834636+0000","last_active":"2024-09-10T19:17:10.166971+0000","last_peered":"2024-09-10T19:17:10.166971+0000","last_clean":"2024-09-10T19:17:10.166971+0000","last_became_active":"2024-09-10T19:16:00.834150+0000","last_became_peered":"2024-09-10T19:16:00.834150+0000","last_unstale":"2024-09-10T19:17:10.166971+0000","last_undegraded":"2024-09-10T19:17:10.166971+0000","last_fullsized":"2024-09-10T19:17:10.166971+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:04.114396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309145+0000","last_change":"2024-09-10T19:16:02.842367+0000","last_active":"2024-09-10T19:16:09.309145+0000","last_peered":"2024-09-10T19:16:09.309145+0000","last_clean":"2024-09-10T19:16:09.309145+0000","last_became_active":"2024-09-10T19:16:02.842061+0000","last_became_peered":"2024-09-10T19:16:02.842061+0000","last_unstale":"2024-09-10T19:16:09.309145+0000","last_undegraded":"2024-09-10T19:16:09.309145+0000","last_fullsized":"2024-09-10T19:16:09.309145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:34.471651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"60'28","reported_seq":80,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.166972+0000","last_change":"2024-09-10T19:15:58.835178+0000","last_active":"2024-09-10T19:17:10.166972+0000","last_peered":"2024-09-10T19:17:10.166972+0000","last_clean":"2024-09-10T19:17:10.166972+0000","last_became_active":"2024-09-10T19:15:58.834893+0000","last_became_peered":"2024-09-10T19:15:58.834893+0000","last_unstale":"2024-09-10T19:17:10.166972+0000","last_undegraded":"2024-09-10T19:17:10.166972+0000","last_fullsized":"2024-09-10T19:17:10.166972+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:36.797750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680691+0000","last_change":"2024-09-10T19:15:56.814418+0000","last_active":"2024-09-10T19:16:09.680691+0000","last_peered":"2024-09-10T19:16:09.680691+0000","last_clean":"2024-09-10T19:16:09.680691+0000","last_became_active":"2024-09-10T19:15:56.813534+0000","last_became_peered":"2024-09-10T19:15:56.813534+0000","last_unstale":"2024-09-10T19:16:09.680691+0000","last_undegraded":"2024-09-10T19:16:09.680691+0000","last_fullsized":"2024-09-10T19:16:09.680691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:32.916762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309723+0000","last_change":"2024-09-10T19:16:00.832390+0000","last_active":"2024-09-10T19:16:09.309723+0000","last_peered":"2024-09-10T19:16:09.309723+0000","last_clean":"2024-09-10T19:16:09.309723+0000","last_became_active":"2024-09-10T19:16:00.832129+0000","last_became_peered":"2024-09-10T19:16:00.832129+0000","last_unstale":"2024-09-10T19:16:09.309723+0000","last_undegraded":"2024-09-10T19:16:09.309723+0000","last_fullsized":"2024-09-10T19:16:09.309723+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:51.232485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310441+0000","last_change":"2024-09-10T19:16:02.832266+0000","last_active":"2024-09-10T19:16:09.310441+0000","last_peered":"2024-09-10T19:16:09.310441+0000","last_clean":"2024-09-10T19:16:09.310441+0000","last_became_active":"2024-09-10T19:16:02.830372+0000","last_became_peered":"2024-09-10T19:16:02.830372+0000","last_unstale":"2024-09-10T19:16:09.310441+0000","last_undegraded":"2024-09-10T19:16:09.310441+0000","last_fullsized":"2024-09-10T19:16:09.310441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:53.660233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"60'13","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680116+0000","last_change":"2024-09-10T19:15:58.829728+0000","last_active":"2024-09-10T19:16:09.680116+0000","last_peered":"2024-09-10T19:16:09.680116+0000","last_clean":"2024-09-10T19:16:09.680116+0000","last_became_active":"2024-09-10T19:15:58.829395+0000","last_became_peered":"2024-09-10T19:15:58.829395+0000","last_unstale":"2024-09-10T19:16:09.680116+0000","last_undegraded":"2024-09-10T19:16:09.680116+0000","last_fullsized":"2024-09-10T19:16:09.680116+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:05.159670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"61'5","reported_seq":101,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:17.969263+0000","last_change":"2024-09-10T19:16:00.820995+0000","last_active":"2024-09-10T19:17:17.969263+0000","last_peered":"2024-09-10T19:17:17.969263+0000","last_clean":"2024-09-10T19:17:17.969263+0000","last_became_active":"2024-09-10T19:15:56.816813+0000","last_became_peered":"2024-09-10T19:15:56.816813+0000","last_unstale":"2024-09-10T19:17:17.969263+0000","last_undegraded":"2024-09-10T19:17:17.969263+0000","last_fullsized":"2024-09-10T19:17:17.969263+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:01.342709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033535900000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"53'1","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678418+0000","last_change":"2024-09-10T19:15:56.805620+0000","last_active":"2024-09-10T19:16:09.678418+0000","last_peered":"2024-09-10T19:16:09.678418+0000","last_clean":"2024-09-10T19:16:09.678418+0000","last_became_active":"2024-09-10T19:15:56.805336+0000","last_became_peered":"2024-09-10T19:15:56.805336+0000","last_unstale":"2024-09-10T19:16:09.678418+0000","last_undegraded":"2024-09-10T19:16:09.678418+0000","last_fullsized":"2024-09-10T19:16:09.678418+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:13.400160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678783+0000","last_change":"2024-09-10T19:16:00.831347+0000","last_active":"2024-09-10T19:16:09.678783+0000","last_peered":"2024-09-10T19:16:09.678783+0000","last_clean":"2024-09-10T19:16:09.678783+0000","last_became_active":"2024-09-10T19:16:00.831001+0000","last_became_peered":"2024-09-10T19:16:00.831001+0000","last_unstale":"2024-09-10T19:16:09.678783+0000","last_undegraded":"2024-09-10T19:16:09.678783+0000","last_fullsized":"2024-09-10T19:16:09.678783+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:25:49.082418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679035+0000","last_change":"2024-09-10T19:16:03.116809+0000","last_active":"2024-09-10T19:16:09.679035+0000","last_peered":"2024-09-10T19:16:09.679035+0000","last_clean":"2024-09-10T19:16:09.679035+0000","last_became_active":"2024-09-10T19:16:03.116497+0000","last_became_peered":"2024-09-10T19:16:03.116497+0000","last_unstale":"2024-09-10T19:16:09.679035+0000","last_undegraded":"2024-09-10T19:16:09.679035+0000","last_fullsized":"2024-09-10T19:16:09.679035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:48.716725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"60'12","reported_seq":34,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311220+0000","last_change":"2024-09-10T19:15:58.830601+0000","last_active":"2024-09-10T19:16:09.311220+0000","last_peered":"2024-09-10T19:16:09.311220+0000","last_clean":"2024-09-10T19:16:09.311220+0000","last_became_active":"2024-09-10T19:15:58.830343+0000","last_became_peered":"2024-09-10T19:15:58.830343+0000","last_unstale":"2024-09-10T19:16:09.311220+0000","last_undegraded":"2024-09-10T19:16:09.311220+0000","last_fullsized":"2024-09-10T19:16:09.311220+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:15.556782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"55'1","reported_seq":24,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309724+0000","last_change":"2024-09-10T19:16:00.825660+0000","last_active":"2024-09-10T19:16:09.309724+0000","last_peered":"2024-09-10T19:16:09.309724+0000","last_clean":"2024-09-10T19:16:09.309724+0000","last_became_active":"2024-09-10T19:15:56.813686+0000","last_became_peered":"2024-09-10T19:15:56.813686+0000","last_unstale":"2024-09-10T19:16:09.309724+0000","last_undegraded":"2024-09-10T19:16:09.309724+0000","last_fullsized":"2024-09-10T19:16:09.309724+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:51.761088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080658900000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679425+0000","last_change":"2024-09-10T19:15:56.818639+0000","last_active":"2024-09-10T19:16:09.679425+0000","last_peered":"2024-09-10T19:16:09.679425+0000","last_clean":"2024-09-10T19:16:09.679425+0000","last_became_active":"2024-09-10T19:15:56.818379+0000","last_became_peered":"2024-09-10T19:15:56.818379+0000","last_unstale":"2024-09-10T19:16:09.679425+0000","last_undegraded":"2024-09-10T19:16:09.679425+0000","last_fullsized":"2024-09-10T19:16:09.679425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:08.135051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681019+0000","last_change":"2024-09-10T19:16:00.836434+0000","last_active":"2024-09-10T19:16:09.681019+0000","last_peered":"2024-09-10T19:16:09.681019+0000","last_clean":"2024-09-10T19:16:09.681019+0000","last_became_active":"2024-09-10T19:16:00.836171+0000","last_became_peered":"2024-09-10T19:16:00.836171+0000","last_unstale":"2024-09-10T19:16:09.681019+0000","last_undegraded":"2024-09-10T19:16:09.681019+0000","last_fullsized":"2024-09-10T19:16:09.681019+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:15.914055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679554+0000","last_change":"2024-09-10T19:16:02.839239+0000","last_active":"2024-09-10T19:16:09.679554+0000","last_peered":"2024-09-10T19:16:09.679554+0000","last_clean":"2024-09-10T19:16:09.679554+0000","last_became_active":"2024-09-10T19:16:02.838960+0000","last_became_peered":"2024-09-10T19:16:02.838960+0000","last_unstale":"2024-09-10T19:16:09.679554+0000","last_undegraded":"2024-09-10T19:16:09.679554+0000","last_fullsized":"2024-09-10T19:16:09.679554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:39.695018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"60'14","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679740+0000","last_change":"2024-09-10T19:15:59.479170+0000","last_active":"2024-09-10T19:16:09.679740+0000","last_peered":"2024-09-10T19:16:09.679740+0000","last_clean":"2024-09-10T19:16:09.679740+0000","last_became_active":"2024-09-10T19:15:59.478866+0000","last_became_peered":"2024-09-10T19:15:59.478866+0000","last_unstale":"2024-09-10T19:16:09.679740+0000","last_undegraded":"2024-09-10T19:16:09.679740+0000","last_fullsized":"2024-09-10T19:16:09.679740+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:05.509410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"57'2","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310489+0000","last_change":"2024-09-10T19:16:00.823451+0000","last_active":"2024-09-10T19:16:09.310489+0000","last_peered":"2024-09-10T19:16:09.310489+0000","last_clean":"2024-09-10T19:16:09.310489+0000","last_became_active":"2024-09-10T19:15:56.812606+0000","last_became_peered":"2024-09-10T19:15:56.812606+0000","last_unstale":"2024-09-10T19:16:09.310489+0000","last_undegraded":"2024-09-10T19:16:09.310489+0000","last_fullsized":"2024-09-10T19:16:09.310489+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:11:04.194548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030027740000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679251+0000","last_change":"2024-09-10T19:15:56.809970+0000","last_active":"2024-09-10T19:16:09.679251+0000","last_peered":"2024-09-10T19:16:09.679251+0000","last_clean":"2024-09-10T19:16:09.679251+0000","last_became_active":"2024-09-10T19:15:56.809700+0000","last_became_peered":"2024-09-10T19:15:56.809700+0000","last_unstale":"2024-09-10T19:16:09.679251+0000","last_undegraded":"2024-09-10T19:16:09.679251+0000","last_fullsized":"2024-09-10T19:16:09.679251+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:56.042615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"62'79","reported_seq":52,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:16:35.751619+0000","last_change":"2024-09-10T19:15:46.309327+0000","last_active":"2024-09-10T19:16:35.751619+0000","last_peered":"2024-09-10T19:16:35.751619+0000","last_clean":"2024-09-10T19:16:35.751619+0000","last_became_active":"2024-09-10T19:15:46.304277+0000","last_became_peered":"2024-09-10T19:15:46.304277+0000","last_unstale":"2024-09-10T19:16:35.751619+0000","last_undegraded":"2024-09-10T19:16:35.751619+0000","last_fullsized":"2024-09-10T19:16:35.751619+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:13:50.198934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:13:50.198934+0000","last_clean_scrub_stamp":"2024-09-10T19:13:50.198934+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:16.862540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679546+0000","last_change":"2024-09-10T19:16:00.834559+0000","last_active":"2024-09-10T19:16:09.679546+0000","last_peered":"2024-09-10T19:16:09.679546+0000","last_clean":"2024-09-10T19:16:09.679546+0000","last_became_active":"2024-09-10T19:16:00.833709+0000","last_became_peered":"2024-09-10T19:16:00.833709+0000","last_unstale":"2024-09-10T19:16:09.679546+0000","last_undegraded":"2024-09-10T19:16:09.679546+0000","last_fullsized":"2024-09-10T19:16:09.679546+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:05:00.147941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680562+0000","last_change":"2024-09-10T19:16:02.836722+0000","last_active":"2024-09-10T19:16:09.680562+0000","last_peered":"2024-09-10T19:16:09.680562+0000","last_clean":"2024-09-10T19:16:09.680562+0000","last_became_active":"2024-09-10T19:16:02.836420+0000","last_became_peered":"2024-09-10T19:16:02.836420+0000","last_unstale":"2024-09-10T19:16:09.680562+0000","last_undegraded":"2024-09-10T19:16:09.680562+0000","last_fullsized":"2024-09-10T19:16:09.680562+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:00.547439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"60'19","reported_seq":47,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680914+0000","last_change":"2024-09-10T19:15:59.478109+0000","last_active":"2024-09-10T19:16:09.680914+0000","last_peered":"2024-09-10T19:16:09.680914+0000","last_clean":"2024-09-10T19:16:09.680914+0000","last_became_active":"2024-09-10T19:15:59.477833+0000","last_became_peered":"2024-09-10T19:15:59.477833+0000","last_unstale":"2024-09-10T19:16:09.680914+0000","last_undegraded":"2024-09-10T19:16:09.680914+0000","last_fullsized":"2024-09-10T19:16:09.680914+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:27:25.856436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679224+0000","last_change":"2024-09-10T19:15:56.807617+0000","last_active":"2024-09-10T19:16:09.679224+0000","last_peered":"2024-09-10T19:16:09.679224+0000","last_clean":"2024-09-10T19:16:09.679224+0000","last_became_active":"2024-09-10T19:15:56.807350+0000","last_became_peered":"2024-09-10T19:15:56.807350+0000","last_unstale":"2024-09-10T19:16:09.679224+0000","last_undegraded":"2024-09-10T19:16:09.679224+0000","last_fullsized":"2024-09-10T19:16:09.679224+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:44.422976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678604+0000","last_change":"2024-09-10T19:16:00.835608+0000","last_active":"2024-09-10T19:16:09.678604+0000","last_peered":"2024-09-10T19:16:09.678604+0000","last_clean":"2024-09-10T19:16:09.678604+0000","last_became_active":"2024-09-10T19:16:00.835272+0000","last_became_peered":"2024-09-10T19:16:00.835272+0000","last_unstale":"2024-09-10T19:16:09.678604+0000","last_undegraded":"2024-09-10T19:16:09.678604+0000","last_fullsized":"2024-09-10T19:16:09.678604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:49.984604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679500+0000","last_change":"2024-09-10T19:16:02.839251+0000","last_active":"2024-09-10T19:16:09.679500+0000","last_peered":"2024-09-10T19:16:09.679500+0000","last_clean":"2024-09-10T19:16:09.679500+0000","last_became_active":"2024-09-10T19:16:02.839050+0000","last_became_peered":"2024-09-10T19:16:02.839050+0000","last_unstale":"2024-09-10T19:16:09.679500+0000","last_undegraded":"2024-09-10T19:16:09.679500+0000","last_fullsized":"2024-09-10T19:16:09.679500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:38.771883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311307+0000","last_change":"2024-09-10T19:15:58.831395+0000","last_active":"2024-09-10T19:16:09.311307+0000","last_peered":"2024-09-10T19:16:09.311307+0000","last_clean":"2024-09-10T19:16:09.311307+0000","last_became_active":"2024-09-10T19:15:58.831143+0000","last_became_peered":"2024-09-10T19:15:58.831143+0000","last_unstale":"2024-09-10T19:16:09.311307+0000","last_undegraded":"2024-09-10T19:16:09.311307+0000","last_fullsized":"2024-09-10T19:16:09.311307+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:50.024094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680091+0000","last_change":"2024-09-10T19:15:56.818922+0000","last_active":"2024-09-10T19:16:09.680091+0000","last_peered":"2024-09-10T19:16:09.680091+0000","last_clean":"2024-09-10T19:16:09.680091+0000","last_became_active":"2024-09-10T19:15:56.818614+0000","last_became_peered":"2024-09-10T19:15:56.818614+0000","last_unstale":"2024-09-10T19:16:09.680091+0000","last_undegraded":"2024-09-10T19:16:09.680091+0000","last_fullsized":"2024-09-10T19:16:09.680091+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:12:05.135056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678469+0000","last_change":"2024-09-10T19:16:00.840244+0000","last_active":"2024-09-10T19:16:09.678469+0000","last_peered":"2024-09-10T19:16:09.678469+0000","last_clean":"2024-09-10T19:16:09.678469+0000","last_became_active":"2024-09-10T19:16:00.839917+0000","last_became_peered":"2024-09-10T19:16:00.839917+0000","last_unstale":"2024-09-10T19:16:09.678469+0000","last_undegraded":"2024-09-10T19:16:09.678469+0000","last_fullsized":"2024-09-10T19:16:09.678469+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:10.279003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309530+0000","last_change":"2024-09-10T19:16:02.841310+0000","last_active":"2024-09-10T19:16:09.309530+0000","last_peered":"2024-09-10T19:16:09.309530+0000","last_clean":"2024-09-10T19:16:09.309530+0000","last_became_active":"2024-09-10T19:16:02.840875+0000","last_became_peered":"2024-09-10T19:16:02.840875+0000","last_unstale":"2024-09-10T19:16:09.309530+0000","last_undegraded":"2024-09-10T19:16:09.309530+0000","last_fullsized":"2024-09-10T19:16:09.309530+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:08.582420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680703+0000","last_change":"2024-09-10T19:15:59.478048+0000","last_active":"2024-09-10T19:16:09.680703+0000","last_peered":"2024-09-10T19:16:09.680703+0000","last_clean":"2024-09-10T19:16:09.680703+0000","last_became_active":"2024-09-10T19:15:59.477665+0000","last_became_peered":"2024-09-10T19:15:59.477665+0000","last_unstale":"2024-09-10T19:16:09.680703+0000","last_undegraded":"2024-09-10T19:16:09.680703+0000","last_fullsized":"2024-09-10T19:16:09.680703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:26.018844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678909+0000","last_change":"2024-09-10T19:15:56.810583+0000","last_active":"2024-09-10T19:16:09.678909+0000","last_peered":"2024-09-10T19:16:09.678909+0000","last_clean":"2024-09-10T19:16:09.678909+0000","last_became_active":"2024-09-10T19:15:56.810308+0000","last_became_peered":"2024-09-10T19:15:56.810308+0000","last_unstale":"2024-09-10T19:16:09.678909+0000","last_undegraded":"2024-09-10T19:16:09.678909+0000","last_fullsized":"2024-09-10T19:16:09.678909+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:58.888976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167209+0000","last_change":"2024-09-10T19:16:00.832364+0000","last_active":"2024-09-10T19:17:10.167209+0000","last_peered":"2024-09-10T19:17:10.167209+0000","last_clean":"2024-09-10T19:17:10.167209+0000","last_became_active":"2024-09-10T19:16:00.832040+0000","last_became_peered":"2024-09-10T19:16:00.832040+0000","last_unstale":"2024-09-10T19:17:10.167209+0000","last_undegraded":"2024-09-10T19:17:10.167209+0000","last_fullsized":"2024-09-10T19:17:10.167209+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:50.602033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680147+0000","last_change":"2024-09-10T19:16:03.115997+0000","last_active":"2024-09-10T19:16:09.680147+0000","last_peered":"2024-09-10T19:16:09.680147+0000","last_clean":"2024-09-10T19:16:09.680147+0000","last_became_active":"2024-09-10T19:16:03.115726+0000","last_became_peered":"2024-09-10T19:16:03.115726+0000","last_unstale":"2024-09-10T19:16:09.680147+0000","last_undegraded":"2024-09-10T19:16:09.680147+0000","last_fullsized":"2024-09-10T19:16:09.680147+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:34:43.776994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308390+0000","last_change":"2024-09-10T19:15:58.830722+0000","last_active":"2024-09-10T19:16:09.308390+0000","last_peered":"2024-09-10T19:16:09.308390+0000","last_clean":"2024-09-10T19:16:09.308390+0000","last_became_active":"2024-09-10T19:15:58.830401+0000","last_became_peered":"2024-09-10T19:15:58.830401+0000","last_unstale":"2024-09-10T19:16:09.308390+0000","last_undegraded":"2024-09-10T19:16:09.308390+0000","last_fullsized":"2024-09-10T19:16:09.308390+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:28.117209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"53'2","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308985+0000","last_change":"2024-09-10T19:15:56.806373+0000","last_active":"2024-09-10T19:16:09.308985+0000","last_peered":"2024-09-10T19:16:09.308985+0000","last_clean":"2024-09-10T19:16:09.308985+0000","last_became_active":"2024-09-10T19:15:56.806129+0000","last_became_peered":"2024-09-10T19:15:56.806129+0000","last_unstale":"2024-09-10T19:16:09.308985+0000","last_undegraded":"2024-09-10T19:16:09.308985+0000","last_fullsized":"2024-09-10T19:16:09.308985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:21.665650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678345+0000","last_change":"2024-09-10T19:16:00.831974+0000","last_active":"2024-09-10T19:16:09.678345+0000","last_peered":"2024-09-10T19:16:09.678345+0000","last_clean":"2024-09-10T19:16:09.678345+0000","last_became_active":"2024-09-10T19:16:00.831691+0000","last_became_peered":"2024-09-10T19:16:00.831691+0000","last_unstale":"2024-09-10T19:16:09.678345+0000","last_undegraded":"2024-09-10T19:16:09.678345+0000","last_fullsized":"2024-09-10T19:16:09.678345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:03.901713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309514+0000","last_change":"2024-09-10T19:16:02.839599+0000","last_active":"2024-09-10T19:16:09.309514+0000","last_peered":"2024-09-10T19:16:09.309514+0000","last_clean":"2024-09-10T19:16:09.309514+0000","last_became_active":"2024-09-10T19:16:02.838951+0000","last_became_peered":"2024-09-10T19:16:02.838951+0000","last_unstale":"2024-09-10T19:16:09.309514+0000","last_undegraded":"2024-09-10T19:16:09.309514+0000","last_fullsized":"2024-09-10T19:16:09.309514+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:05.429846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311475+0000","last_change":"2024-09-10T19:15:59.477764+0000","last_active":"2024-09-10T19:16:09.311475+0000","last_peered":"2024-09-10T19:16:09.311475+0000","last_clean":"2024-09-10T19:16:09.311475+0000","last_became_active":"2024-09-10T19:15:59.477508+0000","last_became_peered":"2024-09-10T19:15:59.477508+0000","last_unstale":"2024-09-10T19:16:09.311475+0000","last_undegraded":"2024-09-10T19:16:09.311475+0000","last_fullsized":"2024-09-10T19:16:09.311475+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:01.854980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678583+0000","last_change":"2024-09-10T19:15:56.811444+0000","last_active":"2024-09-10T19:16:09.678583+0000","last_peered":"2024-09-10T19:16:09.678583+0000","last_clean":"2024-09-10T19:16:09.678583+0000","last_became_active":"2024-09-10T19:15:56.811188+0000","last_became_peered":"2024-09-10T19:15:56.811188+0000","last_unstale":"2024-09-10T19:16:09.678583+0000","last_undegraded":"2024-09-10T19:16:09.678583+0000","last_fullsized":"2024-09-10T19:16:09.678583+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:15:16.906924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310898+0000","last_change":"2024-09-10T19:16:00.825360+0000","last_active":"2024-09-10T19:16:09.310898+0000","last_peered":"2024-09-10T19:16:09.310898+0000","last_clean":"2024-09-10T19:16:09.310898+0000","last_became_active":"2024-09-10T19:16:00.825076+0000","last_became_peered":"2024-09-10T19:16:00.825076+0000","last_unstale":"2024-09-10T19:16:09.310898+0000","last_undegraded":"2024-09-10T19:16:09.310898+0000","last_fullsized":"2024-09-10T19:16:09.310898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:22.386847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678724+0000","last_change":"2024-09-10T19:16:02.836433+0000","last_active":"2024-09-10T19:16:09.678724+0000","last_peered":"2024-09-10T19:16:09.678724+0000","last_clean":"2024-09-10T19:16:09.678724+0000","last_became_active":"2024-09-10T19:16:02.836159+0000","last_became_peered":"2024-09-10T19:16:02.836159+0000","last_unstale":"2024-09-10T19:16:09.678724+0000","last_undegraded":"2024-09-10T19:16:09.678724+0000","last_fullsized":"2024-09-10T19:16:09.678724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:44:09.020929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678996+0000","last_change":"2024-09-10T19:15:56.807417+0000","last_active":"2024-09-10T19:16:09.678996+0000","last_peered":"2024-09-10T19:16:09.678996+0000","last_clean":"2024-09-10T19:16:09.678996+0000","last_became_active":"2024-09-10T19:15:56.807162+0000","last_became_peered":"2024-09-10T19:15:56.807162+0000","last_unstale":"2024-09-10T19:16:09.678996+0000","last_undegraded":"2024-09-10T19:16:09.678996+0000","last_fullsized":"2024-09-10T19:16:09.678996+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:29:47.474664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"60'6","reported_seq":25,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311049+0000","last_change":"2024-09-10T19:15:59.477898+0000","last_active":"2024-09-10T19:16:09.311049+0000","last_peered":"2024-09-10T19:16:09.311049+0000","last_clean":"2024-09-10T19:16:09.311049+0000","last_became_active":"2024-09-10T19:15:59.477405+0000","last_became_peered":"2024-09-10T19:15:59.477405+0000","last_unstale":"2024-09-10T19:16:09.311049+0000","last_undegraded":"2024-09-10T19:16:09.311049+0000","last_fullsized":"2024-09-10T19:16:09.311049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:40.664642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679557+0000","last_change":"2024-09-10T19:16:00.837493+0000","last_active":"2024-09-10T19:16:09.679557+0000","last_peered":"2024-09-10T19:16:09.679557+0000","last_clean":"2024-09-10T19:16:09.679557+0000","last_became_active":"2024-09-10T19:16:00.837210+0000","last_became_peered":"2024-09-10T19:16:00.837210+0000","last_unstale":"2024-09-10T19:16:09.679557+0000","last_undegraded":"2024-09-10T19:16:09.679557+0000","last_fullsized":"2024-09-10T19:16:09.679557+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:43.335638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678952+0000","last_change":"2024-09-10T19:16:03.116153+0000","last_active":"2024-09-10T19:16:09.678952+0000","last_peered":"2024-09-10T19:16:09.678952+0000","last_clean":"2024-09-10T19:16:09.678952+0000","last_became_active":"2024-09-10T19:16:03.115871+0000","last_became_peered":"2024-09-10T19:16:03.115871+0000","last_unstale":"2024-09-10T19:16:09.678952+0000","last_undegraded":"2024-09-10T19:16:09.678952+0000","last_fullsized":"2024-09-10T19:16:09.678952+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:00:21.986445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311160+0000","last_change":"2024-09-10T19:15:59.480380+0000","last_active":"2024-09-10T19:16:09.311160+0000","last_peered":"2024-09-10T19:16:09.311160+0000","last_clean":"2024-09-10T19:16:09.311160+0000","last_became_active":"2024-09-10T19:15:59.480104+0000","last_became_peered":"2024-09-10T19:15:59.480104+0000","last_unstale":"2024-09-10T19:16:09.311160+0000","last_undegraded":"2024-09-10T19:16:09.311160+0000","last_fullsized":"2024-09-10T19:16:09.311160+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:18.922754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680621+0000","last_change":"2024-09-10T19:15:56.814668+0000","last_active":"2024-09-10T19:16:09.680621+0000","last_peered":"2024-09-10T19:16:09.680621+0000","last_clean":"2024-09-10T19:16:09.680621+0000","last_became_active":"2024-09-10T19:15:56.813933+0000","last_became_peered":"2024-09-10T19:15:56.813933+0000","last_unstale":"2024-09-10T19:16:09.680621+0000","last_undegraded":"2024-09-10T19:16:09.680621+0000","last_fullsized":"2024-09-10T19:16:09.680621+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:51.925652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167231+0000","last_change":"2024-09-10T19:16:00.837931+0000","last_active":"2024-09-10T19:17:10.167231+0000","last_peered":"2024-09-10T19:17:10.167231+0000","last_clean":"2024-09-10T19:17:10.167231+0000","last_became_active":"2024-09-10T19:16:00.837619+0000","last_became_peered":"2024-09-10T19:16:00.837619+0000","last_unstale":"2024-09-10T19:17:10.167231+0000","last_undegraded":"2024-09-10T19:17:10.167231+0000","last_fullsized":"2024-09-10T19:17:10.167231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:53.040495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310850+0000","last_change":"2024-09-10T19:16:02.842576+0000","last_active":"2024-09-10T19:16:09.310850+0000","last_peered":"2024-09-10T19:16:09.310850+0000","last_clean":"2024-09-10T19:16:09.310850+0000","last_became_active":"2024-09-10T19:16:02.842284+0000","last_became_peered":"2024-09-10T19:16:02.842284+0000","last_unstale":"2024-09-10T19:16:09.310850+0000","last_undegraded":"2024-09-10T19:16:09.310850+0000","last_fullsized":"2024-09-10T19:16:09.310850+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:25.991343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680877+0000","last_change":"2024-09-10T19:15:59.480057+0000","last_active":"2024-09-10T19:16:09.680877+0000","last_peered":"2024-09-10T19:16:09.680877+0000","last_clean":"2024-09-10T19:16:09.680877+0000","last_became_active":"2024-09-10T19:15:59.479711+0000","last_became_peered":"2024-09-10T19:15:59.479711+0000","last_unstale":"2024-09-10T19:16:09.680877+0000","last_undegraded":"2024-09-10T19:16:09.680877+0000","last_fullsized":"2024-09-10T19:16:09.680877+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:46.572648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311530+0000","last_change":"2024-09-10T19:15:56.814802+0000","last_active":"2024-09-10T19:16:09.311530+0000","last_peered":"2024-09-10T19:16:09.311530+0000","last_clean":"2024-09-10T19:16:09.311530+0000","last_became_active":"2024-09-10T19:15:56.814329+0000","last_became_peered":"2024-09-10T19:15:56.814329+0000","last_unstale":"2024-09-10T19:16:09.311530+0000","last_undegraded":"2024-09-10T19:16:09.311530+0000","last_fullsized":"2024-09-10T19:16:09.311530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:47.101659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"60'8","reported_seq":38,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167039+0000","last_change":"2024-09-10T19:16:00.836128+0000","last_active":"2024-09-10T19:17:10.167039+0000","last_peered":"2024-09-10T19:17:10.167039+0000","last_clean":"2024-09-10T19:17:10.167039+0000","last_became_active":"2024-09-10T19:16:00.835790+0000","last_became_peered":"2024-09-10T19:16:00.835790+0000","last_unstale":"2024-09-10T19:17:10.167039+0000","last_undegraded":"2024-09-10T19:17:10.167039+0000","last_fullsized":"2024-09-10T19:17:10.167039+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:48.515902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308663+0000","last_change":"2024-09-10T19:16:02.832727+0000","last_active":"2024-09-10T19:16:09.308663+0000","last_peered":"2024-09-10T19:16:09.308663+0000","last_clean":"2024-09-10T19:16:09.308663+0000","last_became_active":"2024-09-10T19:16:02.832469+0000","last_became_peered":"2024-09-10T19:16:02.832469+0000","last_unstale":"2024-09-10T19:16:09.308663+0000","last_undegraded":"2024-09-10T19:16:09.308663+0000","last_fullsized":"2024-09-10T19:16:09.308663+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:41.568602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"60'9","reported_seq":32,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679532+0000","last_change":"2024-09-10T19:15:58.835886+0000","last_active":"2024-09-10T19:16:09.679532+0000","last_peered":"2024-09-10T19:16:09.679532+0000","last_clean":"2024-09-10T19:16:09.679532+0000","last_became_active":"2024-09-10T19:15:58.835522+0000","last_became_peered":"2024-09-10T19:15:58.835522+0000","last_unstale":"2024-09-10T19:16:09.679532+0000","last_undegraded":"2024-09-10T19:16:09.679532+0000","last_fullsized":"2024-09-10T19:16:09.679532+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:06.306813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679541+0000","last_change":"2024-09-10T19:15:56.814106+0000","last_active":"2024-09-10T19:16:09.679541+0000","last_peered":"2024-09-10T19:16:09.679541+0000","last_clean":"2024-09-10T19:16:09.679541+0000","last_became_active":"2024-09-10T19:15:56.813740+0000","last_became_peered":"2024-09-10T19:15:56.813740+0000","last_unstale":"2024-09-10T19:16:09.679541+0000","last_undegraded":"2024-09-10T19:16:09.679541+0000","last_fullsized":"2024-09-10T19:16:09.679541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:16.778568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309587+0000","last_change":"2024-09-10T19:16:00.828756+0000","last_active":"2024-09-10T19:16:09.309587+0000","last_peered":"2024-09-10T19:16:09.309587+0000","last_clean":"2024-09-10T19:16:09.309587+0000","last_became_active":"2024-09-10T19:16:00.828481+0000","last_became_peered":"2024-09-10T19:16:00.828481+0000","last_unstale":"2024-09-10T19:16:09.309587+0000","last_undegraded":"2024-09-10T19:16:09.309587+0000","last_fullsized":"2024-09-10T19:16:09.309587+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:28.472480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309145+0000","last_change":"2024-09-10T19:16:02.843426+0000","last_active":"2024-09-10T19:16:09.309145+0000","last_peered":"2024-09-10T19:16:09.309145+0000","last_clean":"2024-09-10T19:16:09.309145+0000","last_became_active":"2024-09-10T19:16:02.843163+0000","last_became_peered":"2024-09-10T19:16:02.843163+0000","last_unstale":"2024-09-10T19:16:09.309145+0000","last_undegraded":"2024-09-10T19:16:09.309145+0000","last_fullsized":"2024-09-10T19:16:09.309145+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:16.365335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308578+0000","last_change":"2024-09-10T19:15:58.831059+0000","last_active":"2024-09-10T19:16:09.308578+0000","last_peered":"2024-09-10T19:16:09.308578+0000","last_clean":"2024-09-10T19:16:09.308578+0000","last_became_active":"2024-09-10T19:15:58.830700+0000","last_became_peered":"2024-09-10T19:15:58.830700+0000","last_unstale":"2024-09-10T19:16:09.308578+0000","last_undegraded":"2024-09-10T19:16:09.308578+0000","last_fullsized":"2024-09-10T19:16:09.308578+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:37.622626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680062+0000","last_change":"2024-09-10T19:15:56.814096+0000","last_active":"2024-09-10T19:16:09.680062+0000","last_peered":"2024-09-10T19:16:09.680062+0000","last_clean":"2024-09-10T19:16:09.680062+0000","last_became_active":"2024-09-10T19:15:56.813796+0000","last_became_peered":"2024-09-10T19:15:56.813796+0000","last_unstale":"2024-09-10T19:16:09.680062+0000","last_undegraded":"2024-09-10T19:16:09.680062+0000","last_fullsized":"2024-09-10T19:16:09.680062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:49.847594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679383+0000","last_change":"2024-09-10T19:16:00.832798+0000","last_active":"2024-09-10T19:16:09.679383+0000","last_peered":"2024-09-10T19:16:09.679383+0000","last_clean":"2024-09-10T19:16:09.679383+0000","last_became_active":"2024-09-10T19:16:00.832514+0000","last_became_peered":"2024-09-10T19:16:00.832514+0000","last_unstale":"2024-09-10T19:16:09.679383+0000","last_undegraded":"2024-09-10T19:16:09.679383+0000","last_fullsized":"2024-09-10T19:16:09.679383+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:13.788951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310043+0000","last_change":"2024-09-10T19:16:02.841281+0000","last_active":"2024-09-10T19:16:09.310043+0000","last_peered":"2024-09-10T19:16:09.310043+0000","last_clean":"2024-09-10T19:16:09.310043+0000","last_became_active":"2024-09-10T19:16:02.840820+0000","last_became_peered":"2024-09-10T19:16:02.840820+0000","last_unstale":"2024-09-10T19:16:09.310043+0000","last_undegraded":"2024-09-10T19:16:09.310043+0000","last_fullsized":"2024-09-10T19:16:09.310043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:19.856229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"60'4","reported_seq":22,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310255+0000","last_change":"2024-09-10T19:15:58.831283+0000","last_active":"2024-09-10T19:16:09.310255+0000","last_peered":"2024-09-10T19:16:09.310255+0000","last_clean":"2024-09-10T19:16:09.310255+0000","last_became_active":"2024-09-10T19:15:58.830910+0000","last_became_peered":"2024-09-10T19:15:58.830910+0000","last_unstale":"2024-09-10T19:16:09.310255+0000","last_undegraded":"2024-09-10T19:16:09.310255+0000","last_fullsized":"2024-09-10T19:16:09.310255+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:43.005321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680383+0000","last_change":"2024-09-10T19:15:56.812953+0000","last_active":"2024-09-10T19:16:09.680383+0000","last_peered":"2024-09-10T19:16:09.680383+0000","last_clean":"2024-09-10T19:16:09.680383+0000","last_became_active":"2024-09-10T19:15:56.812669+0000","last_became_peered":"2024-09-10T19:15:56.812669+0000","last_unstale":"2024-09-10T19:16:09.680383+0000","last_undegraded":"2024-09-10T19:16:09.680383+0000","last_fullsized":"2024-09-10T19:16:09.680383+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:58.987254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678763+0000","last_change":"2024-09-10T19:16:00.833086+0000","last_active":"2024-09-10T19:16:09.678763+0000","last_peered":"2024-09-10T19:16:09.678763+0000","last_clean":"2024-09-10T19:16:09.678763+0000","last_became_active":"2024-09-10T19:16:00.832727+0000","last_became_peered":"2024-09-10T19:16:00.832727+0000","last_unstale":"2024-09-10T19:16:09.678763+0000","last_undegraded":"2024-09-10T19:16:09.678763+0000","last_fullsized":"2024-09-10T19:16:09.678763+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:03.555485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678827+0000","last_change":"2024-09-10T19:16:02.833685+0000","last_active":"2024-09-10T19:16:09.678827+0000","last_peered":"2024-09-10T19:16:09.678827+0000","last_clean":"2024-09-10T19:16:09.678827+0000","last_became_active":"2024-09-10T19:16:02.833410+0000","last_became_peered":"2024-09-10T19:16:02.833410+0000","last_unstale":"2024-09-10T19:16:09.678827+0000","last_undegraded":"2024-09-10T19:16:09.678827+0000","last_fullsized":"2024-09-10T19:16:09.678827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:44.049592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310366+0000","last_change":"2024-09-10T19:15:59.477484+0000","last_active":"2024-09-10T19:16:09.310366+0000","last_peered":"2024-09-10T19:16:09.310366+0000","last_clean":"2024-09-10T19:16:09.310366+0000","last_became_active":"2024-09-10T19:15:59.477187+0000","last_became_peered":"2024-09-10T19:15:59.477187+0000","last_unstale":"2024-09-10T19:16:09.310366+0000","last_undegraded":"2024-09-10T19:16:09.310366+0000","last_fullsized":"2024-09-10T19:16:09.310366+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:47.343734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680901+0000","last_change":"2024-09-10T19:15:56.813276+0000","last_active":"2024-09-10T19:16:09.680901+0000","last_peered":"2024-09-10T19:16:09.680901+0000","last_clean":"2024-09-10T19:16:09.680901+0000","last_became_active":"2024-09-10T19:15:56.813031+0000","last_became_peered":"2024-09-10T19:15:56.813031+0000","last_unstale":"2024-09-10T19:16:09.680901+0000","last_undegraded":"2024-09-10T19:16:09.680901+0000","last_fullsized":"2024-09-10T19:16:09.680901+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:13.966067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.678487+0000","last_change":"2024-09-10T19:16:00.832082+0000","last_active":"2024-09-10T19:16:09.678487+0000","last_peered":"2024-09-10T19:16:09.678487+0000","last_clean":"2024-09-10T19:16:09.678487+0000","last_became_active":"2024-09-10T19:16:00.831524+0000","last_became_peered":"2024-09-10T19:16:00.831524+0000","last_unstale":"2024-09-10T19:16:09.678487+0000","last_undegraded":"2024-09-10T19:16:09.678487+0000","last_fullsized":"2024-09-10T19:16:09.678487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:02.109784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309425+0000","last_change":"2024-09-10T19:16:03.115843+0000","last_active":"2024-09-10T19:16:09.309425+0000","last_peered":"2024-09-10T19:16:09.309425+0000","last_clean":"2024-09-10T19:16:09.309425+0000","last_became_active":"2024-09-10T19:16:03.115430+0000","last_became_peered":"2024-09-10T19:16:03.115430+0000","last_unstale":"2024-09-10T19:16:09.309425+0000","last_undegraded":"2024-09-10T19:16:09.309425+0000","last_fullsized":"2024-09-10T19:16:09.309425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:05.719257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679465+0000","last_change":"2024-09-10T19:16:02.837432+0000","last_active":"2024-09-10T19:16:09.679465+0000","last_peered":"2024-09-10T19:16:09.679465+0000","last_clean":"2024-09-10T19:16:09.679465+0000","last_became_active":"2024-09-10T19:16:02.837160+0000","last_became_peered":"2024-09-10T19:16:02.837160+0000","last_unstale":"2024-09-10T19:16:09.679465+0000","last_undegraded":"2024-09-10T19:16:09.679465+0000","last_fullsized":"2024-09-10T19:16:09.679465+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:58.230781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680777+0000","last_change":"2024-09-10T19:15:56.808289+0000","last_active":"2024-09-10T19:16:09.680777+0000","last_peered":"2024-09-10T19:16:09.680777+0000","last_clean":"2024-09-10T19:16:09.680777+0000","last_became_active":"2024-09-10T19:15:56.807387+0000","last_became_peered":"2024-09-10T19:15:56.807387+0000","last_unstale":"2024-09-10T19:16:09.680777+0000","last_undegraded":"2024-09-10T19:16:09.680777+0000","last_fullsized":"2024-09-10T19:16:09.680777+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:48.709035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"60'10","reported_seq":31,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311107+0000","last_change":"2024-09-10T19:15:59.479758+0000","last_active":"2024-09-10T19:16:09.311107+0000","last_peered":"2024-09-10T19:16:09.311107+0000","last_clean":"2024-09-10T19:16:09.311107+0000","last_became_active":"2024-09-10T19:15:59.479500+0000","last_became_peered":"2024-09-10T19:15:59.479500+0000","last_unstale":"2024-09-10T19:16:09.311107+0000","last_undegraded":"2024-09-10T19:16:09.311107+0000","last_fullsized":"2024-09-10T19:16:09.311107+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:48.212860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"60'8","reported_seq":41,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-10T19:17:10.167460+0000","last_change":"2024-09-10T19:16:00.835173+0000","last_active":"2024-09-10T19:17:10.167460+0000","last_peered":"2024-09-10T19:17:10.167460+0000","last_clean":"2024-09-10T19:17:10.167460+0000","last_became_active":"2024-09-10T19:16:00.834417+0000","last_became_peered":"2024-09-10T19:16:00.834417+0000","last_unstale":"2024-09-10T19:17:10.167460+0000","last_undegraded":"2024-09-10T19:17:10.167460+0000","last_fullsized":"2024-09-10T19:17:10.167460+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:36.906340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"60'1","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679756+0000","last_change":"2024-09-10T19:16:02.842552+0000","last_active":"2024-09-10T19:16:09.679756+0000","last_peered":"2024-09-10T19:16:09.679756+0000","last_clean":"2024-09-10T19:16:09.679756+0000","last_became_active":"2024-09-10T19:16:02.842238+0000","last_became_peered":"2024-09-10T19:16:02.842238+0000","last_unstale":"2024-09-10T19:16:09.679756+0000","last_undegraded":"2024-09-10T19:16:09.679756+0000","last_fullsized":"2024-09-10T19:16:09.679756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:42:54.648225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310635+0000","last_change":"2024-09-10T19:15:56.811746+0000","last_active":"2024-09-10T19:16:09.310635+0000","last_peered":"2024-09-10T19:16:09.310635+0000","last_clean":"2024-09-10T19:16:09.310635+0000","last_became_active":"2024-09-10T19:15:56.811503+0000","last_became_peered":"2024-09-10T19:15:56.811503+0000","last_unstale":"2024-09-10T19:16:09.310635+0000","last_undegraded":"2024-09-10T19:16:09.310635+0000","last_fullsized":"2024-09-10T19:16:09.310635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:33.995220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"60'11","reported_seq":35,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.680823+0000","last_change":"2024-09-10T19:15:58.830917+0000","last_active":"2024-09-10T19:16:09.680823+0000","last_peered":"2024-09-10T19:16:09.680823+0000","last_clean":"2024-09-10T19:16:09.680823+0000","last_became_active":"2024-09-10T19:15:58.830623+0000","last_became_peered":"2024-09-10T19:15:58.830623+0000","last_unstale":"2024-09-10T19:16:09.680823+0000","last_undegraded":"2024-09-10T19:16:09.680823+0000","last_fullsized":"2024-09-10T19:16:09.680823+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:32.904029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.311010+0000","last_change":"2024-09-10T19:16:00.835141+0000","last_active":"2024-09-10T19:16:09.311010+0000","last_peered":"2024-09-10T19:16:09.311010+0000","last_clean":"2024-09-10T19:16:09.311010+0000","last_became_active":"2024-09-10T19:16:00.834727+0000","last_became_peered":"2024-09-10T19:16:00.834727+0000","last_unstale":"2024-09-10T19:16:09.311010+0000","last_undegraded":"2024-09-10T19:16:09.311010+0000","last_fullsized":"2024-09-10T19:16:09.311010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:52.987521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308691+0000","last_change":"2024-09-10T19:16:03.115482+0000","last_active":"2024-09-10T19:16:09.308691+0000","last_peered":"2024-09-10T19:16:09.308691+0000","last_clean":"2024-09-10T19:16:09.308691+0000","last_became_active":"2024-09-10T19:16:03.115201+0000","last_became_peered":"2024-09-10T19:16:03.115201+0000","last_unstale":"2024-09-10T19:16:09.308691+0000","last_undegraded":"2024-09-10T19:16:09.308691+0000","last_fullsized":"2024-09-10T19:16:09.308691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:34.609845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.681000+0000","last_change":"2024-09-10T19:15:56.810432+0000","last_active":"2024-09-10T19:16:09.681000+0000","last_peered":"2024-09-10T19:16:09.681000+0000","last_clean":"2024-09-10T19:16:09.681000+0000","last_became_active":"2024-09-10T19:15:56.808882+0000","last_became_peered":"2024-09-10T19:15:56.808882+0000","last_unstale":"2024-09-10T19:16:09.681000+0000","last_undegraded":"2024-09-10T19:16:09.681000+0000","last_fullsized":"2024-09-10T19:16:09.681000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:18:17.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"60'15","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679036+0000","last_change":"2024-09-10T19:15:58.832344+0000","last_active":"2024-09-10T19:16:09.679036+0000","last_peered":"2024-09-10T19:16:09.679036+0000","last_clean":"2024-09-10T19:16:09.679036+0000","last_became_active":"2024-09-10T19:15:58.832022+0000","last_became_peered":"2024-09-10T19:15:58.832022+0000","last_unstale":"2024-09-10T19:16:09.679036+0000","last_undegraded":"2024-09-10T19:16:09.679036+0000","last_fullsized":"2024-09-10T19:16:09.679036+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:17.309755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.679949+0000","last_change":"2024-09-10T19:16:00.833127+0000","last_active":"2024-09-10T19:16:09.679949+0000","last_peered":"2024-09-10T19:16:09.679949+0000","last_clean":"2024-09-10T19:16:09.679949+0000","last_became_active":"2024-09-10T19:16:00.832866+0000","last_became_peered":"2024-09-10T19:16:00.832866+0000","last_unstale":"2024-09-10T19:16:09.679949+0000","last_undegraded":"2024-09-10T19:16:09.679949+0000","last_fullsized":"2024-09-10T19:16:09.679949+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:24:36.173793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.310929+0000","last_change":"2024-09-10T19:16:03.115614+0000","last_active":"2024-09-10T19:16:09.310929+0000","last_peered":"2024-09-10T19:16:09.310929+0000","last_clean":"2024-09-10T19:16:09.310929+0000","last_became_active":"2024-09-10T19:16:03.115024+0000","last_became_peered":"2024-09-10T19:16:03.115024+0000","last_unstale":"2024-09-10T19:16:09.310929+0000","last_undegraded":"2024-09-10T19:16:09.310929+0000","last_fullsized":"2024-09-10T19:16:09.310929+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:16:01.817698+0000","last_clean_scrub_stamp":"2024-09-10T19:16:01.817698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:36.701997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309799+0000","last_change":"2024-09-10T19:15:56.810022+0000","last_active":"2024-09-10T19:16:09.309799+0000","last_peered":"2024-09-10T19:16:09.309799+0000","last_clean":"2024-09-10T19:16:09.309799+0000","last_became_active":"2024-09-10T19:15:56.809776+0000","last_became_peered":"2024-09-10T19:15:56.809776+0000","last_unstale":"2024-09-10T19:16:09.309799+0000","last_undegraded":"2024-09-10T19:16:09.309799+0000","last_fullsized":"2024-09-10T19:16:09.309799+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:55.790342+0000","last_clean_scrub_stamp":"2024-09-10T19:15:55.790342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:06.855548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"60'12","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.309486+0000","last_change":"2024-09-10T19:15:58.830445+0000","last_active":"2024-09-10T19:16:09.309486+0000","last_peered":"2024-09-10T19:16:09.309486+0000","last_clean":"2024-09-10T19:16:09.309486+0000","last_became_active":"2024-09-10T19:15:58.830053+0000","last_became_peered":"2024-09-10T19:15:58.830053+0000","last_unstale":"2024-09-10T19:16:09.309486+0000","last_undegraded":"2024-09-10T19:16:09.309486+0000","last_fullsized":"2024-09-10T19:16:09.309486+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:57.807009+0000","last_clean_scrub_stamp":"2024-09-10T19:15:57.807009+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:51.165963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T19:16:09.308636+0000","last_change":"2024-09-10T19:16:00.833524+0000","last_active":"2024-09-10T19:16:09.308636+0000","last_peered":"2024-09-10T19:16:09.308636+0000","last_clean":"2024-09-10T19:16:09.308636+0000","last_became_active":"2024-09-10T19:16:00.833210+0000","last_became_peered":"2024-09-10T19:16:00.833210+0000","last_unstale":"2024-09-10T19:16:09.308636+0000","last_undegraded":"2024-09-10T19:16:09.308636+0000","last_fullsized":"2024-09-10T19:16:09.308636+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T19:15:59.810128+0000","last_clean_scrub_stamp":"2024-09-10T19:15:59.810128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:12.600185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":70,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397525,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1216512,"data_stored":825817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.56000000000000005}]},{"osd":3,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.503},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.35899999999999999}]},{"osd":6,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.40100000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1204224,"data_stored":825544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.499}]},{"osd":1,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58899999999999997}]},{"osd":2,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.629}]},{"osd":3,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.40000000000000002}]},{"osd":5,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.36699999999999999}]},{"osd":7,"last update":"Tue Sep 10 19:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.438}]}]},{"osd":5,"up_from":37,"seq":158913789983,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297948,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447204,"statfs":{"total":95995035648,"available":95689936896,"internally_reserved":0,"allocated":606208,"data_stored":234007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":0.69399999999999995,"15min":0.58799999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":18.146999999999998,"5min":18.146999999999998,"15min":18.146999999999998},"last":0.439},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54500000000000004,"15min":0.55200000000000005},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.67100000000000004,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.496},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.53900000000000003},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53300000000000003,"15min":0.53000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55400000000000005,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.72799999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.34000000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.39000000000000001,"15min":0.375},"min":{"1min":0.29299999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.505}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.372},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.33600000000000002}]},{"osd":7,"last update":"Tue Sep 10 19:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40300000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447184,"statfs":{"total":95995035648,"available":95689916416,"internally_reserved":0,"allocated":626688,"data_stored":234439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.32200000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58699999999999997}]},{"osd":1,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.33600000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60099999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48499999999999999,"15min":0.48399999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.50800000000000001,"15min":0.51000000000000001},"min":{"1min":0.29599999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55800000000000005}]},{"osd":3,"last update":"Tue Sep 10 19:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.52800000000000002,"15min":0.53100000000000003},"min":{"1min":0.34300000000000003,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.251,"15min":0.251},"max":{"1min":0.67800000000000005,"5min":0.753,"15min":0.753},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.373,"15min":0.36499999999999999},"min":{"1min":0.23699999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.379,"15min":0.377},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56599999999999995,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.52700000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.57799999999999996}]},{"osd":7,"last update":"Tue Sep 10 19:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.42299999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297984,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447168,"statfs":{"total":95995035648,"available":95689900032,"internally_reserved":0,"allocated":643072,"data_stored":235139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.307,"5min":0.35499999999999998,"15min":0.36399999999999999},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.40899999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.375,"15min":0.379},"min":{"1min":0.26400000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.47499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38600000000000001,"15min":0.39400000000000002},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.373,"15min":0.38100000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.42499999999999999,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.36899999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:17:15 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.498,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.315},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.52500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.26500000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:16:38 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52200000000000002,"15min":0.52100000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55000000000000004,"15min":0.54300000000000004},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.38700000000000001}]},{"osd":5,"last update":"Tue Sep 10 19:17:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.745,"5min":1.3,"15min":1.3},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:46 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.56799999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":234219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.36499999999999999,"15min":0.36099999999999999},"min":{"1min":0.221,"5min":0.214,"15min":0.214},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.36699999999999999,"15min":0.35999999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.373}]},{"osd":1,"last update":"Tue Sep 10 19:16:56 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.377,"15min":0.373},"min":{"1min":0.26100000000000001,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.374,"5min":0.38700000000000001,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.59799999999999998,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.29499999999999998}]},{"osd":3,"last update":"Tue Sep 10 19:16:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.42399999999999999,"15min":0.42799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.45500000000000002,"15min":0.46100000000000002},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64300000000000002,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Sep 10 19:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.65000000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67600000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.308}]},{"osd":5,"last update":"Tue Sep 10 19:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54200000000000004,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.67200000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.71499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.33900000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:22 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.443},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.34699999999999998}]},{"osd":7,"last update":"Tue Sep 10 19:16:49 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.433},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":235285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 19:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.32200000000000001,"5min":0.32100000000000001,"15min":0.311},"min":{"1min":0.23100000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.34300000000000003},"min":{"1min":0.23799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Sep 10 19:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.374},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.374},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.36699999999999999,"15min":0.36299999999999999},"min":{"1min":0.26100000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.40799999999999997}]},{"osd":3,"last update":"Tue Sep 10 19:16:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.40500000000000003,"15min":0.41199999999999998},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51000000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Sep 10 19:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70099999999999996,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46300000000000002}]},{"osd":6,"last update":"Tue Sep 10 19:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.433},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:45 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.38700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298544,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446608,"statfs":{"total":95995035648,"available":95689326592,"internally_reserved":0,"allocated":1220608,"data_stored":825876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 19:16:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.33400000000000002,"15min":0.318},"min":{"1min":0.247,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53500000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.249},{"interface":"front","average":{"1min":0.378,"5min":0.34599999999999997,"15min":0.32400000000000001},"min":{"1min":0.23300000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.57199999999999995,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Sep 10 19:16:52 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.378,"15min":0.378},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.45800000000000002,"5min":0.63,"15min":0.63},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.39000000000000001,"15min":0.38800000000000001},"min":{"1min":0.24299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.47699999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.314}]},{"osd":3,"last update":"Tue Sep 10 19:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.41699999999999998,"15min":0.436},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.441,"15min":0.46100000000000002},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.505,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.28199999999999997}]},{"osd":4,"last update":"Tue Sep 10 19:16:40 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56299999999999994,"15min":0.56000000000000005},"min":{"1min":0.36799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70699999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Sep 10 19:17:01 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78800000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.84599999999999997,"15min":0.88800000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":6.9509999999999996,"15min":6.9509999999999996},"last":0.375}]},{"osd":6,"last update":"Tue Sep 10 19:16:25 2024","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.976,"5min":6.976,"15min":6.976},"last":0.40100000000000002},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":12.502000000000001,"5min":12.502000000000001,"15min":12.502000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Tue Sep 10 19:16:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T19:17:23.498 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T19:17:23.498 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T19:17:23.498 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T19:17:23.499 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T19:17:23.738 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:24.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-mon[33096]: from='client.14736 v1:172.21.15.115:0/4195170906' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:24.383 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 podman[63308]: 2024-09-10 19:17:24.343717667 +0000 UTC m=+0.132071624 container create f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , distribution-scope=public, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, release=239.1651231664, description=Ceph Grafana Container, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-10T19:17:24.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:24 smithi115 ceph-mon[28968]: from='client.14736 v1:172.21.15.115:0/4195170906' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:24.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:24 smithi115 ceph-mon[39520]: from='client.14736 v1:172.21.15.115:0/4195170906' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T19:17:24.738 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 podman[63308]: 2024-09-10 19:17:24.579528208 +0000 UTC m=+0.367882137 container init f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, vendor=Red Hat, Inc., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, maintainer=Paul Cuzner , version=8.5, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, distribution-scope=public, vcs-type=git, description=Ceph Grafana Container, architecture=x86_64) 2024-09-10T19:17:24.738 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 podman[63308]: 2024-09-10 19:17:24.582678197 +0000 UTC m=+0.371032122 container start f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, release=239.1651231664, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, distribution-scope=public) 2024-09-10T19:17:24.738 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 bash[63308]: f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff 2024-09-10T19:17:24.738 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 systemd[1]: Started Ceph grafana.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:24.739 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740026444Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740205941Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740225633Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740233513Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740239611Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T19:17:24.989 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740245773Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740252202Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740258195Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740264133Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740269915Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740275535Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740281366Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740287122Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=settings t=2024-09-10T19:17:24.740293034Z level=info msg="App mode production" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore t=2024-09-10T19:17:24.74034173Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore t=2024-09-10T19:17:24.740357768Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.741222061Z level=info msg="Starting DB migrations" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.741306331Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T19:17:24.990 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.741830946Z level=info msg="Executing migration" id="create user table" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.742260327Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.743092653Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.744066713Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.744852891Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.745830483Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T19:17:24.991 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.746585344Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.747049499Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.749925113Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.752643465Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.75544316Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.758051406Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.76095145Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.763945806Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.766719662Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.769428478Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.772177018Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.775000469Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.777731895Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T19:17:24.992 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.780539656Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.783291559Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.786026568Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.786439633Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.789509591Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.792264385Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.795041206Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.797867563Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.800510929Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.801001981Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.80382029Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.806529575Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T19:17:24.993 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.809189932Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.809899387Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.812477185Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.815145102Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.817857785Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.818225846Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.820913017Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.821264281Z level=info msg="Executing migration" id="create star table" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.823561207Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.824043938Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.826740646Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.829399321Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.830266531Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T19:17:24.994 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.830990962Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.831934736Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.832579931Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.833414998Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.833742529Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.834235618Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.835083198Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.835882567Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.836683442Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.837659488Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.838423367Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.839230649Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T19:17:24.995 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.840195757Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T19:17:24.996 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.840927125Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T19:17:24.996 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.841898237Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.842629113Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.843440682Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.844380027Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.845083019Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.846055596Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.846759313Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.847633447Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.850054773Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.852767743Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T19:17:24.997 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.855583738Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.858380586Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.860893696Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.863624233Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.866201773Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.866796751Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.869591346Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.872329526Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.875042696Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.877640813Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.88028556Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.882813437Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.8853292Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.888010133Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.890689571Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T19:17:24.998 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.893559059Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.896250431Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.898936719Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.90158542Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.904092174Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.906638747Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.907444491Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.908355232Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.90904813Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.909900733Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.910732455Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.911702587Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.912334715Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.912808292Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T19:17:24.999 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.913609012Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.914508389Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.915760437Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.916686189Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.917381292Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.918342188Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.919001482Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.919856739Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.920659218Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.921610574Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.922319097Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.923149704Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.924060013Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.924954418Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T19:17:25.000 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.925856589Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.926742776Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.927460123Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.928281927Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.929181828Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.929861692Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.930858931Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.931578218Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.932560446Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.933325417Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.933726254Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.934798847Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.935571635Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T19:17:25.001 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.936025128Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.936440905Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.936860937Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.937182744Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.93751476Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.937786059Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.938356053Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.93885008Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.939191517Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.939702007Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.940299336Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.9406831Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.941065869Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T19:17:25.002 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.941450744Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.941897108Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.942325805Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.942718453Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.943026884Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.943288013Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.943814881Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.944394573Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.944684585Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.945084019Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.945519618Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.945791832Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T19:17:25.003 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.94618778Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.946593714Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.947188719Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.94749035Z level=info msg="Executing migration" id="create session table" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.947936228Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.948230482Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.948510004Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.948903839Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.949282354Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.949539454Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.949803142Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.950110692Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.950390828Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T19:17:25.004 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.950775981Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.951076006Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.951628549Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.951951543Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.95252645Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.953124747Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.953423663Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.953868137Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.954327294Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.955170012Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.957764255Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.958373464Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.958773704Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T19:17:25.005 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.959166156Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.959581382Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.959980672Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.961414266Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.961802232Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.962223539Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.96251236Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.962889623Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.963271175Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.963845723Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.964463839Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.96512486Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.965729557Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.966145092Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T19:17:25.006 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.966430367Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.966688113Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.967077341Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.967484165Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.96784248Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.968239302Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.970655175Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.971398987Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.973475556Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.976187771Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T19:17:25.007 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.979054096Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.979502014Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.982225715Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.98496471Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.987638286Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.988103963Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.990816015Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.993523192Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.996213719Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T19:17:25.008 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.996640136Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T19:17:25.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-mon[33096]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:25.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.240 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:24.99928265Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.001823462Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.004556463Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.004867095Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.005701678Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.006549016Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T19:17:25.241 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.00740013Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.008252355Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.009077621Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.011978143Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.01489064Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.015283344Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.018088105Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.020925781Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.023608895Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.023943322Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.026932103Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.029612184Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.032214484Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T19:17:25.242 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.032672473Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.03564227Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.038307733Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.041046435Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.041402966Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.044219338Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.044704164Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.047111241Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.049680494Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.050305028Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.05295521Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.053618355Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.055532495Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.058212055Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T19:17:25.243 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.058655893Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T19:17:25.244 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.061329283Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T19:17:25.244 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.063856383Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T19:17:25.244 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.066536583Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.067083418Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.069820838Z level=info msg="Executing migration" id="create team table" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.072483074Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.074915036Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.075432358Z level=info msg="Executing migration" id="create team member table" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.078245144Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.080987995Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.081678421Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.082173831Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.082965504Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T19:17:25.245 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.083745879Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.084566665Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.085047616Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.085865376Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.086863193Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.087556275Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.088112716Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.089074025Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.08976579Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.090682605Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.091337945Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.092228597Z level=info msg="Executing migration" id="create tag table" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.092872994Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T19:17:25.246 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.093349132Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.094239873Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.09512598Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.09595885Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.097877253Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.098881133Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.099608443Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.100601162Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.101385531Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.10222652Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.103171243Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.10389861Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.104792636Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.105625741Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T19:17:25.247 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.106479727Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.107346908Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.108189332Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.109049615Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.10998288Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.112443964Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.115119466Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.117985953Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.12058762Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.123461058Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.126369495Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.129126644Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.131869473Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T19:17:25.248 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.134622185Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.137163022Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.139964104Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.142726209Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.145469415Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.148348236Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.150970162Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.153855484Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.156622923Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.159392528Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.162312608Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.167605125Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.168439391Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.171197729Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T19:17:25.249 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.173964884Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.176414145Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.179073467Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.17953002Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.182177409Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.182622862Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.184978065Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.187731877Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.188292239Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.190876329Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.193662044Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.196356355Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.199762703Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T19:17:25.250 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.203080442Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.204005677Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.204664538Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.205452833Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.206400141Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.207215195Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.207919265Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.208485924Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.209213832Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.210192156Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.211162407Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.211991519Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.212436221Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.215184653Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T19:17:25.251 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.216029567Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.217974367Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.220677678Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.221586525Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.224437967Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.227010701Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.229747981Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.23023796Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.232922933Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.233238421Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.235950799Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.23681983Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.239552625Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T19:17:25.252 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.242125098Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[28968]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.429 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[39520]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:25.430 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.430 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.430 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.430 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:25 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:25.504 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.244773267Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T19:17:25.504 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.245101662Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.247888731Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.250514917Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.253102651Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.253983465Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.256663857Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.257579607Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.258419844Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.258876495Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.259726953Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.260558211Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.261378019Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.261796709Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.262605028Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T19:17:25.505 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.263558388Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T19:17:25.506 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.263983961Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T19:17:25.506 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.264476621Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T19:17:25.506 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.26456438Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T19:17:25.506 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.26522332Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T19:17:25.506 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.265599542Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.266115464Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.266245613Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.266376744Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.266910447Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.267380655Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.268341178Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.269015579Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.269830654Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.270325855Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.271121572Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.27153828Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.272040274Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T19:17:25.507 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.272949194Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.277435785Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.278367324Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.279234941Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.283502725Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.287694564Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.288538978Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.289457484Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.289838248Z level=info msg="Executing migration" id="create permission table" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.290286518Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.2911844Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T19:17:25.508 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.291891954Z level=info msg="Executing migration" id="create role table" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.292896482Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.29394095Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.296616966Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.299249793Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.302130159Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.304801899Z level=info msg="Executing migration" id="create team role table" 2024-09-10T19:17:25.509 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.307386812Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.310306971Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.31306573Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.31349794Z level=info msg="Executing migration" id="create user role table" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.316446938Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.319136035Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.321842365Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.322275837Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.324970298Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.327625578Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.330319667Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T19:17:25.510 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.331383043Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.334186966Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.337013562Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.337459154Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.337881508Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.338325304Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.338823339Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.339787472Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.340205434Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.340658459Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.340968383Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.341479633Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T19:17:25.511 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.341957948Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T19:17:25.512 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.342352736Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T19:17:25.512 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.342662161Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T19:17:25.512 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.343082492Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.343508669Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.343994024Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.344980178Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.345291211Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.345759672Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.346252783Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.346734224Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.347760827Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.348249472Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.348689167Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.348788167Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:17:25.513 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.349076855Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.349161731Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.350893982Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.353689479Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.35622204Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.358827581Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.361508375Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.364108218Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.366680542Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.367196784Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.369843051Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.370466731Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.372392083Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T19:17:25.514 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.375300503Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.378470858Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.381220424Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.383688944Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.386322048Z level=info msg="Executing migration" id="add share column" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.387356597Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.388205682Z level=info msg="Executing migration" id="create file table" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.389004231Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.389840294Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.390575116Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.391024268Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.391849443Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T19:17:25.515 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.392692796Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.393161273Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.393935307Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.39439066Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.395462309Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.396293961Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.396756318Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.397544851Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.398317377Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.398689966Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.399710011Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.400759002Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.401541011Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T19:17:25.516 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.411403543Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.412202292Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.412657851Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.41604097Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.416799128Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.417945436Z level=info msg="Executing migration" id="create folder table" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.418361653Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.419281387Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.419950367Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.420932456Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=migrator t=2024-09-10T19:17:25.42149354Z level=info msg="migrations completed" performed=485 skipped=0 duration=680.217716ms 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore t=2024-09-10T19:17:25.421894108Z level=info msg="Created default organization" 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=plugin.loader t=2024-09-10T19:17:25.476505005Z level=info msg="Plugin registered" pluginID=input 2024-09-10T19:17:25.517 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=plugin.loader t=2024-09-10T19:17:25.503018589Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T19:17:25.518 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=plugin.loader t=2024-09-10T19:17:25.503052687Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T19:17:25.518 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=secrets t=2024-09-10T19:17:25.503120201Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T19:17:25.518 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=query_data t=2024-09-10T19:17:25.50673129Z level=info msg="Query Service initialization" 2024-09-10T19:17:25.605 INFO:teuthology.orchestra.run.smithi115.stdout: 2024-09-10T19:17:25.605 INFO:teuthology.orchestra.run.smithi115.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T19:17:25.882 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=live.push_http t=2024-09-10T19:17:25.510003145Z level=info msg="Live Push Gateway initialization" 2024-09-10T19:17:26.296 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T19:17:26.296 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T19:17:26.296 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T19:17:26.305 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T19:17:26.306 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 -- bash -c 'set -ex 2024-09-10T19:17:26.306 DEBUG:teuthology.orchestra.run.smithi115:> for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:17:26.306 DEBUG:teuthology.orchestra.run.smithi115:> do 2024-09-10T19:17:26.306 DEBUG:teuthology.orchestra.run.smithi115:> echo "rotating key for $f" 2024-09-10T19:17:26.306 DEBUG:teuthology.orchestra.run.smithi115:> K=$(ceph auth get-key $f) 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> NK="$K" 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> ceph orch daemon rotate-key $f 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> while [ "$K" == "$NK" ]; do 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> sleep 5 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> NK=$(ceph auth get-key $f) 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> done 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> done 2024-09-10T19:17:26.307 DEBUG:teuthology.orchestra.run.smithi115:> ' 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[28968]: Deploying daemon node-exporter.a on smithi115 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3846135352' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[39520]: Deploying daemon node-exporter.a on smithi115 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:26.533 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:26 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3846135352' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:17:26.579 INFO:teuthology.orchestra.run.smithi115.stderr:Inferring config /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T19:17:26.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-mon[33096]: Deploying daemon node-exporter.a on smithi115 2024-09-10T19:17:26.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:26.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3846135352' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=infra.usagestats.collector t=2024-09-10T19:17:26.242862285Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=provisioning.datasources t=2024-09-10T19:17:26.24326378Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=provisioning.datasources t=2024-09-10T19:17:26.27676717Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=provisioning.datasources t=2024-09-10T19:17:26.308255408Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=provisioning.alerting t=2024-09-10T19:17:26.329558331Z level=info msg="starting to provision alerting" 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=provisioning.alerting t=2024-09-10T19:17:26.329577205Z level=info msg="finished to provision alerting" 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=ngalert.state.manager t=2024-09-10T19:17:26.329651332Z level=info msg="Warming state cache for startup" 2024-09-10T19:17:26.633 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=grafanaStorageLogger t=2024-09-10T19:17:26.329731049Z level=info msg="storage starting" 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=http.server t=2024-09-10T19:17:26.331276824Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore.transactions t=2024-09-10T19:17:26.340641178Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=ngalert.state.manager t=2024-09-10T19:17:26.340710879Z level=info msg="State cache has been initialized" states=0 duration=11.058534ms 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=ticker t=2024-09-10T19:17:26.340775922Z level=info msg=starting first_tick=2024-09-10T19:17:30Z 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=ngalert.multiorg.alertmanager t=2024-09-10T19:17:26.340858923Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore.transactions t=2024-09-10T19:17:26.390989006Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:17:26.634 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:17:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a[63323]: logger=sqlstore.transactions t=2024-09-10T19:17:26.401419345Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T19:17:27.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:27 smithi150 ceph-mon[33096]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:27.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:27 smithi115 ceph-mon[28968]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:27.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:27 smithi115 ceph-mon[39520]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:28.612 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.0 2024-09-10T19:17:28.612 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:17:28.612 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.0' 2024-09-10T19:17:28.612 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:28.808 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:28.809 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:28.809 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.0 2024-09-10T19:17:29.094 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.0 on host 'smithi115' 2024-09-10T19:17:29.109 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:29.109 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:29.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[39520]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2584062566' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[28968]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2584062566' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.414 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 bash[72336]: Writing manifest to image destination 2024-09-10T19:17:29.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:29 smithi150 ceph-mon[33096]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:29.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:29 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:29 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2584062566' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:29.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:29 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:29 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:29.913 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 podman[72336]: 2024-09-10 19:17:29.598864355 +0000 UTC m=+3.166298767 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T19:17:29.913 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 podman[72336]: 2024-09-10 19:17:29.706006998 +0000 UTC m=+3.273441483 container create 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:30.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[39520]: from='client.14754 v1:172.21.15.115:0/4022082797' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:17:30.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.0 2024-09-10T19:17:30.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[28968]: from='client.14754 v1:172.21.15.115:0/4022082797' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.0 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 podman[72336]: 2024-09-10 19:17:29.950113664 +0000 UTC m=+3.517548077 container init 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 podman[72336]: 2024-09-10 19:17:29.953607974 +0000 UTC m=+3.521042396 container start 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.958Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.958Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.959Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.959Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.959Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.959Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T19:17:30.415 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T19:17:30.416 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T19:17:30.417 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T19:17:30.418 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a[72562]: ts=2024-09-10T19:17:29.960Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:30 smithi115 bash[72336]: 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 2024-09-10T19:17:30.419 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:17:30 smithi115 systemd[1]: Started Ceph node-exporter.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:30.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:30 smithi150 ceph-mon[33096]: from='client.14754 v1:172.21.15.115:0/4022082797' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:17:30.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:30 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.0 2024-09-10T19:17:30.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:30.431 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:31.133 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:31 smithi150 bash[63581]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T19:17:31.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:31 smithi150 ceph-mon[33096]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:31.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:31 smithi150 ceph-mon[33096]: Deploying daemon node-exporter.b on smithi150 2024-09-10T19:17:31.633 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:31 smithi150 bash[63581]: Getting image source signatures 2024-09-10T19:17:31.633 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:31 smithi150 bash[63581]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T19:17:31.633 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:31 smithi150 bash[63581]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T19:17:31.633 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:31 smithi150 bash[63581]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T19:17:31.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:31 smithi115 ceph-mon[28968]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:31.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:31 smithi115 ceph-mon[28968]: Deploying daemon node-exporter.b on smithi150 2024-09-10T19:17:31.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:31 smithi115 ceph-mon[39520]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:31.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:31 smithi115 ceph-mon[39520]: Deploying daemon node-exporter.b on smithi150 2024-09-10T19:17:32.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:32 smithi150 ceph-mon[33096]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:32.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:32 smithi115 ceph-mon[28968]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:32.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:32 smithi115 ceph-mon[39520]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:33.132 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:32 smithi150 bash[63581]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T19:17:33.133 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:32 smithi150 bash[63581]: Writing manifest to image destination 2024-09-10T19:17:33.603 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 podman[63581]: 2024-09-10 19:17:33.285092507 +0000 UTC m=+2.360030219 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T19:17:33.603 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 podman[63581]: 2024-09-10 19:17:33.38354714 +0000 UTC m=+2.458484842 container create b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:17:33.603 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 podman[63581]: 2024-09-10 19:17:33.602220162 +0000 UTC m=+2.677157863 container init b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:17:33.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 podman[63581]: 2024-09-10 19:17:33.605408103 +0000 UTC m=+2.680345804 container start b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.609Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.609Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.609Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.609Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T19:17:33.884 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T19:17:33.885 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T19:17:33.886 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T19:17:33.887 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.610Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.611Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b[63657]: ts=2024-09-10T19:17:33.611Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 bash[63581]: b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b 2024-09-10T19:17:33.888 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:17:33 smithi150 systemd[1]: Started Ceph node-exporter.b for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:33.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T19:17:33.915 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:34.112 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:34.437 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:35.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:35.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:34 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3292769518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T19:17:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3292769518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T19:17:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:35.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:34 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3292769518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:36.119 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:35 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:36.119 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:35 smithi115 ceph-mon[39520]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:17:36.119 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:35 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:36.119 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:35 smithi115 ceph-mon[28968]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:17:36.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:35 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:36.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:35 smithi150 ceph-mon[33096]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T19:17:37.111 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:36 smithi150 ceph-mon[33096]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:37.111 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:36 smithi150 ceph-mon[33096]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:17:37.111 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:36 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:37.111 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:36 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[28968]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[28968]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:17:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:37.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[39520]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[39520]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T19:17:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:36 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.283 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.284 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:17:39.285 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:39 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.439 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:39.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]: dispatch 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.0", "format": "json"}]': finished 2024-09-10T19:17:39.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:39 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:39.747 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[28968]: Rotating authentication key for osd.0 2024-09-10T19:17:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T19:17:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1436619015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[39520]: Rotating authentication key for osd.0 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1436619015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:40.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:40.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:40 smithi150 ceph-mon[33096]: Rotating authentication key for osd.0 2024-09-10T19:17:40.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:40 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.0 on smithi115 2024-09-10T19:17:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1436619015' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[28968]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[28968]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:17:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[28968]: Cluster is now healthy 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[39520]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[39520]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[39520]: Cluster is now healthy 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:41 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:41 smithi150 ceph-mon[33096]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:41 smithi150 ceph-mon[33096]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T19:17:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:41 smithi150 ceph-mon[33096]: Cluster is now healthy 2024-09-10T19:17:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:41 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:41 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:42.383 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:42 smithi115 ceph-mon[28968]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:17:42.383 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:42 smithi115 ceph-mon[28968]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T19:17:42.383 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:42 smithi115 ceph-mon[39520]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:17:42.383 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:42 smithi115 ceph-mon[39520]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T19:17:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:42 smithi150 ceph-mon[33096]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T19:17:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:42 smithi150 ceph-mon[33096]: Reconfiguring daemon alertmanager.a on smithi115 2024-09-10T19:17:43.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:43 smithi150 ceph-mon[33096]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:43.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:43 smithi115 ceph-mon[28968]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:43.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:43 smithi115 ceph-mon[39520]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 54 B/s, 2 objects/s recovering 2024-09-10T19:17:43.663 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:43 smithi115 systemd[1]: Stopping Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:17:43.986 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[71157]: ts=2024-09-10T19:17:43.700Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:17:43.986 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:43 smithi115 podman[73490]: 2024-09-10 19:17:43.730963144 +0000 UTC m=+0.200171371 container died 812b5f2c997530cb8bfa8be1c529b0175109b34de8bff7726d7ef58683730de1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:44.413 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73490]: 2024-09-10 19:17:44.192255113 +0000 UTC m=+0.661463341 container cleanup 812b5f2c997530cb8bfa8be1c529b0175109b34de8bff7726d7ef58683730de1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:44.413 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 bash[73490]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:17:44.749 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:44.749 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73502]: 2024-09-10 19:17:44.468574775 +0000 UTC m=+0.738438544 container remove 812b5f2c997530cb8bfa8be1c529b0175109b34de8bff7726d7ef58683730de1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:44.750 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73502]: 2024-09-10 19:17:44.519140269 +0000 UTC m=+0.789004036 volume remove e6ce332ad55475fbb612179805b452bd2f70ffb5a06c726c8d52de4d2aeed93e 2024-09-10T19:17:44.750 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:17:44.750 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 systemd[1]: Stopped Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:44.750 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 systemd[1]: Starting Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:17:45.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:45.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:45.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:45.061 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73595]: 2024-09-10 19:17:44.8702017 +0000 UTC m=+0.107537177 volume create 281490dd0165cf712cd7bc1c10876a9cdf87a766ac21147ab513533bc3adfa91 2024-09-10T19:17:45.061 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73595]: 2024-09-10 19:17:44.778598567 +0000 UTC m=+0.015934059 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T19:17:45.061 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:44 smithi115 podman[73595]: 2024-09-10 19:17:44.962290468 +0000 UTC m=+0.199625946 container create 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:45.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:45.366 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-mon[28968]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-09-10T19:17:45.366 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1592231430' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:45.367 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-mon[39520]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-09-10T19:17:45.367 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1592231430' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 podman[73595]: 2024-09-10 19:17:45.265471177 +0000 UTC m=+0.502806657 container init 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 podman[73595]: 2024-09-10 19:17:45.268450131 +0000 UTC m=+0.505785610 container start 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.280Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.280Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.281Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.115 port=9094 2024-09-10T19:17:45.367 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.282Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T19:17:45.368 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.306Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T19:17:45.368 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.306Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T19:17:45.368 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.309Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T19:17:45.368 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:45.309Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T19:17:45.368 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 bash[73595]: 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 2024-09-10T19:17:45.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:45 smithi150 ceph-mon[33096]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 926 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-09-10T19:17:45.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1592231430' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:45.659 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:45 smithi115 systemd[1]: Started Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:46.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:46 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:46.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:46.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:46.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:46.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:46.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:46.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:46.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:47 smithi150 ceph-mon[33096]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:17:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:47 smithi150 ceph-mon[33096]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T19:17:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:47 smithi150 ceph-mon[33096]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 280 B/s wr, 7 op/s; 40 B/s, 1 objects/s recovering 2024-09-10T19:17:47.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[28968]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:17:47.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[28968]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T19:17:47.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[28968]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 280 B/s wr, 7 op/s; 40 B/s, 1 objects/s recovering 2024-09-10T19:17:47.663 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:47.283Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000547174s 2024-09-10T19:17:47.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[39520]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T19:17:47.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[39520]: Reconfiguring daemon prometheus.a on smithi150 2024-09-10T19:17:47.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:47 smithi115 ceph-mon[39520]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 280 B/s wr, 7 op/s; 40 B/s, 1 objects/s recovering 2024-09-10T19:17:48.986 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:49.340 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:49 smithi150 ceph-mon[33096]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:49.341 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:49 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:49 smithi115 ceph-mon[28968]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:49 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:49.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:49 smithi115 ceph-mon[39520]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:49.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:49 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:17:50.063 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:50.380 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:50.380 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:50.381 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:50.381 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:50 smithi115 ceph-mon[28968]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:50.381 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:50 smithi115 ceph-mon[39520]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:50.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:50 smithi150 ceph-mon[33096]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 255 B/s wr, 7 op/s; 36 B/s, 1 objects/s recovering 2024-09-10T19:17:51.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3748426616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:51.383 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 systemd[1]: Stopping Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:17:51.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:51 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3748426616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:51 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3748426616' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:51.711 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:17:51.711 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:17:51.711 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:17:51.711 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.392Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.393Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.393Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[61467]: ts=2024-09-10T19:17:51.393Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:17:51.712 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 podman[64561]: 2024-09-10 19:17:51.402867873 +0000 UTC m=+0.125099742 container died e8c9e71ecca18b26ff58ecfda8ae892f6800243f951e5c2b6783c7967c777a6a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:52.044 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 podman[64561]: 2024-09-10 19:17:51.817379919 +0000 UTC m=+0.539611778 container cleanup e8c9e71ecca18b26ff58ecfda8ae892f6800243f951e5c2b6783c7967c777a6a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:52.044 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:51 smithi150 bash[64561]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:17:52.299 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 podman[64572]: 2024-09-10 19:17:52.043689184 +0000 UTC m=+0.642531065 container remove e8c9e71ecca18b26ff58ecfda8ae892f6800243f951e5c2b6783c7967c777a6a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:52.299 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-mon[33096]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:52.618 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T19:17:52.619 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 systemd[1]: Stopped Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:52.619 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.045s CPU time. 2024-09-10T19:17:52.619 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 systemd[1]: Starting Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:17:52.619 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 podman[64666]: 2024-09-10 19:17:52.410601401 +0000 UTC m=+0.016603548 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T19:17:52.619 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 podman[64666]: 2024-09-10 19:17:52.519536942 +0000 UTC m=+0.125539135 container create 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:52.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:52 smithi115 ceph-mon[28968]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:52.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:52 smithi115 ceph-mon[39520]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:53.095 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:52 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:52] ENGINE Bus STOPPING 2024-09-10T19:17:53.095 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:17:53.095 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STOPPED 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 podman[64666]: 2024-09-10 19:17:52.705089931 +0000 UTC m=+0.311092079 container init 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 podman[64666]: 2024-09-10 19:17:52.707737203 +0000 UTC m=+0.313739350 container start 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.731Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.731Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.731Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi150 (none))" 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.731Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.731Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T19:17:53.133 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.732Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.732Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.733Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.733Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.544µs 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.736Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.823µs wal_replay_duration=421.612µs wbl_replay_duration=87ns total_replay_duration=456.213µs 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.737Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.737Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T19:17:53.134 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.737Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T19:17:53.135 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.751Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.195024ms db_storage=718ns remote_storage=1.386µs web_handler=282ns query_engine=483ns scrape=188.785µs scrape_sd=23.538µs notify=21.114µs notify_sd=6.494µs rules=13.663337ms tracing=6.89µs 2024-09-10T19:17:53.135 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.751Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T19:17:53.135 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:17:52.751Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T19:17:53.135 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 bash[64666]: 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 2024-09-10T19:17:53.135 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:17:52 smithi150 systemd[1]: Started Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:17:53.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STARTING 2024-09-10T19:17:53.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Serving on http://:::9283 2024-09-10T19:17:53.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STARTED 2024-09-10T19:17:53.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STOPPING 2024-09-10T19:17:53.803 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:17:53.803 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STOPPED 2024-09-10T19:17:53.803 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STARTING 2024-09-10T19:17:53.803 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Serving on http://:::9283 2024-09-10T19:17:53.804 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STARTED 2024-09-10T19:17:54.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:53 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:54.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:53] ENGINE Bus STOPPING 2024-09-10T19:17:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi115.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi150.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.167 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T19:17:54.167 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.167 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mon.? v1:172.21.15.115:0/3810225676' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi150.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T19:17:54.167 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:54.167 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:53 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:54.600 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:54] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:17:54.601 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:54] ENGINE Bus STOPPED 2024-09-10T19:17:54.601 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:54] ENGINE Bus STARTING 2024-09-10T19:17:54.601 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:54] ENGINE Serving on http://:::9283 2024-09-10T19:17:54.601 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:17:54] ENGINE Bus STARTED 2024-09-10T19:17:55.106 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:54 smithi150 ceph-mon[33096]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:55.106 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:17:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:17:55.133 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-mon[28968]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:55.133 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:54 smithi115 ceph-mon[39520]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:55.382 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:17:55.413 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:17:55.284Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002029088s 2024-09-10T19:17:55.713 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:17:55.713 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:17:55.713 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:17:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:55 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:55 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:55 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/499718197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/499718197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:17:56.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:56.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:55 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/499718197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:17:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:56 smithi150 ceph-mon[33096]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:56 smithi115 ceph-mon[39520]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:56 smithi115 ceph-mon[28968]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:17:58.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:58.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:58.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:57 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:58.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:17:58.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:57 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:17:58.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:17:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:17:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:17:59.234 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:17:58 smithi115 ceph-mon[28968]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:59.234 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:17:58 smithi115 ceph-mon[39520]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:17:59.349 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:17:58 smithi150 ceph-mon[33096]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:00.715 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:00.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:00 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:00.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:00 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:00.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:00 smithi150 ceph-mon[33096]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:01.028 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:01.028 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:01.028 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:01.028 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[28968]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:01.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:01.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:01.029 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:00 smithi115 ceph-mon[39520]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3395888808' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:18:02.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:01 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3395888808' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:18:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3395888808' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:18:02.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:01 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:18:03.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:02 smithi150 ceph-mon[33096]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:03.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:02 smithi115 ceph-mon[39520]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:02 smithi115 ceph-mon[28968]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:04.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:04.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:05.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:04 smithi150 ceph-mon[33096]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:05.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:05.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:04] "GET /metrics HTTP/1.1" 200 219814 "" "Prometheus/2.43.0" 2024-09-10T19:18:05.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:04 smithi115 ceph-mon[39520]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:04 smithi115 ceph-mon[28968]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:06.030 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:06.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:06.341 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:07.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:06 smithi150 ceph-mon[33096]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:07.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:06 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2691512980' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:06 smithi115 ceph-mon[28968]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:06 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2691512980' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:06 smithi115 ceph-mon[39520]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:06 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2691512980' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:09.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:08 smithi150 ceph-mon[33096]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:09.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:09.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:08 smithi115 ceph-mon[28968]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:09.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:08 smithi115 ceph-mon[39520]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:11.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:10 smithi150 ceph-mon[33096]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:10 smithi115 ceph-mon[28968]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:11.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:10 smithi115 ceph-mon[39520]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:11.343 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:11.655 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:11.655 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:11.655 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:11.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:11 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2616921197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:11.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:11 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2616921197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:12.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:11 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2616921197' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:12 smithi150 ceph-mon[33096]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:12 smithi115 ceph-mon[28968]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:12 smithi115 ceph-mon[39520]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:15.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:14 smithi150 ceph-mon[33096]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:15.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:14 smithi115 ceph-mon[28968]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:15.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:14] "GET /metrics HTTP/1.1" 200 219814 "" "Prometheus/2.43.0" 2024-09-10T19:18:15.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:14 smithi115 ceph-mon[39520]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:16.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:15 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:16.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:15 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:16.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:15 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:16.657 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:16.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:16 smithi115 ceph-mon[28968]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:16.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:16 smithi115 ceph-mon[39520]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:16.974 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:16.974 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:16.974 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:17.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:16 smithi150 ceph-mon[33096]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:18.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:17 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3491444794' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:18.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:17 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3491444794' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:18.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:17 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3491444794' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:19.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:18 smithi150 ceph-mon[33096]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:18 smithi115 ceph-mon[28968]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:18 smithi115 ceph-mon[39520]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:20 smithi150 ceph-mon[33096]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:21.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:20 smithi115 ceph-mon[28968]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:21.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:20 smithi115 ceph-mon[39520]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:21.976 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:22.287 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:22.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:23.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:22 smithi150 ceph-mon[33096]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:23.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:22 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4262020680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:23.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:22 smithi115 ceph-mon[28968]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:23.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:22 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4262020680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:23.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:22 smithi115 ceph-mon[39520]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:23.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:22 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4262020680' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:24 smithi150 ceph-mon[33096]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:25.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:24 smithi115 ceph-mon[28968]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:24] "GET /metrics HTTP/1.1" 200 219779 "" "Prometheus/2.43.0" 2024-09-10T19:18:25.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:24 smithi115 ceph-mon[39520]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:25 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:26.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:25 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:26.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:25 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:26 smithi150 ceph-mon[33096]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:26 smithi115 ceph-mon[28968]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:27.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:26 smithi115 ceph-mon[39520]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:27.289 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:27.627 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:27.627 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:27.627 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:28.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:27 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/840971158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:28.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:27 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/840971158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:28.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:27 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/840971158' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:29.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:28 smithi150 ceph-mon[33096]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:29.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:29.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:28 smithi115 ceph-mon[28968]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:29.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:28 smithi115 ceph-mon[39520]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:31.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:30 smithi115 ceph-mon[28968]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:31.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:30 smithi115 ceph-mon[39520]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:31.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:30 smithi150 ceph-mon[33096]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:32.629 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:32.946 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:32.946 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:32.946 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:33.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:32 smithi115 ceph-mon[39520]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:33.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:32 smithi115 ceph-mon[28968]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:33.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:32 smithi150 ceph-mon[33096]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:34.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:33 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2821203206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:34.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:34.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:33 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2821203206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:34.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:34.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:33 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2821203206' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:34.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:35.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:34 smithi150 ceph-mon[33096]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:35.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:34 smithi115 ceph-mon[28968]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:35.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:34] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-10T19:18:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:34 smithi115 ceph-mon[39520]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:36.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:35 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:36.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:35 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:36.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:35 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:36 smithi115 ceph-mon[28968]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:37.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:36 smithi115 ceph-mon[39520]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:37.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:36 smithi150 ceph-mon[33096]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:37.948 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:38.260 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:39.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:39.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:38 smithi150 ceph-mon[33096]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:39.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:38 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1422335102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:39.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:38 smithi115 ceph-mon[28968]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:39.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:38 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1422335102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:39.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:38 smithi115 ceph-mon[39520]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:39.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:38 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1422335102' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:41.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:40 smithi115 ceph-mon[28968]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:41.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:40 smithi115 ceph-mon[39520]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:40 smithi150 ceph-mon[33096]: pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:43.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:42 smithi115 ceph-mon[28968]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:43.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:42 smithi115 ceph-mon[39520]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:43.264 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:43.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:42 smithi150 ceph-mon[33096]: pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:43.572 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:43.572 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:43.573 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:44.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:43 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3276592518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:44.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:43 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3276592518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:44.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:43 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3276592518' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:45.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:44 smithi150 ceph-mon[33096]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:45.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:44 smithi115 ceph-mon[28968]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:45.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:44] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-10T19:18:45.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:44 smithi115 ceph-mon[39520]: pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:46.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:45 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:45 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:46.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:45 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:47.176 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:46 smithi115 ceph-mon[28968]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:47.177 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:46 smithi115 ceph-mon[39520]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:47.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:46 smithi150 ceph-mon[33096]: pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:48.575 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:48.887 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:48.927 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[28968]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2680830480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:49.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[39520]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:49.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:49.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:48 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2680830480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:48 smithi150 ceph-mon[33096]: pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:48 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:18:49.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:48 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2680830480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:50 smithi150 ceph-mon[33096]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:50 smithi115 ceph-mon[28968]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:51.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:50 smithi115 ceph-mon[39520]: pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:53.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:52 smithi150 ceph-mon[33096]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:53.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:52 smithi115 ceph-mon[28968]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:53.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:52 smithi115 ceph-mon[39520]: pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:53.889 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:54.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:18:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:54 smithi150 ceph-mon[33096]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:55.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:54 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/206957477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:55.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:18:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:18:55.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:54 smithi115 ceph-mon[28968]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:55.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:54 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/206957477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:55.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:18:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:54] "GET /metrics HTTP/1.1" 200 219789 "" "Prometheus/2.43.0" 2024-09-10T19:18:55.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:54 smithi115 ceph-mon[39520]: pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:55.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:54 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/206957477' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:18:56.305 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:56.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:56.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:18:57.274 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:56 smithi150 ceph-mon[33096]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:57.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:56 smithi115 ceph-mon[28968]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:57.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:56 smithi115 ceph-mon[39520]: pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:18:58.944 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:18:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:18:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:18:59.205 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:18:59.205 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:18:58 smithi115 ceph-mon[28968]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:59.206 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:18:58 smithi115 ceph-mon[39520]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:18:58 smithi150 ceph-mon[33096]: pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:18:59.520 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:18:59.520 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:18:59.520 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:00 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3557749359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:00.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:00 smithi150 ceph-mon[33096]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:00.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:00 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3557749359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:00.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:00 smithi115 ceph-mon[28968]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:00.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:00 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3557749359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:00.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:00 smithi115 ceph-mon[39520]: pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:01.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:01 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:19:01.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:01 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:19:01.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:01 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:19:02.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:19:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:19:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:19:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:02 smithi150 ceph-mon[33096]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:02.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:19:02.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:19:02.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:19:02.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[28968]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:02.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:19:02.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:19:02.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:19:02.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:02 smithi115 ceph-mon[39520]: pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:04.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:04.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:04.522 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:04.834 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:04.835 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:04.835 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:05.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:04 smithi150 ceph-mon[33096]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:05.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:04 smithi115 ceph-mon[28968]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:05.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:04] "GET /metrics HTTP/1.1" 200 219791 "" "Prometheus/2.43.0" 2024-09-10T19:19:05.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:04 smithi115 ceph-mon[39520]: pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:06.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:05 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/422740947' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:06.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:05 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/422740947' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:05 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/422740947' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:07.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:06 smithi150 ceph-mon[33096]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:06 smithi115 ceph-mon[28968]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:06 smithi115 ceph-mon[39520]: pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:08 smithi150 ceph-mon[33096]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:09.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:09.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:08 smithi115 ceph-mon[28968]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:09.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:08 smithi115 ceph-mon[39520]: pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:09.836 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:10.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:10.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:10.145 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:10 smithi150 ceph-mon[33096]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:10 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/168383141' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:10 smithi115 ceph-mon[28968]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:10 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/168383141' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:11.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:10 smithi115 ceph-mon[39520]: pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:11.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:10 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/168383141' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:12 smithi150 ceph-mon[33096]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:12 smithi115 ceph-mon[28968]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:12 smithi115 ceph-mon[39520]: pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:15.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:14 smithi150 ceph-mon[33096]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:15.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:15.147 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:15.147 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:14] "GET /metrics HTTP/1.1" 200 219791 "" "Prometheus/2.43.0" 2024-09-10T19:19:15.147 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:14 smithi115 ceph-mon[39520]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:15.148 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:14 smithi115 ceph-mon[28968]: pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:15.458 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:16.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:15 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:16.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:15 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/831604661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:16.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:15 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:16.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:15 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/831604661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:16.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:15 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:16.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:15 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/831604661' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:17.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:16 smithi150 ceph-mon[33096]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:17.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:16 smithi115 ceph-mon[28968]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:17.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:16 smithi115 ceph-mon[39520]: pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:19.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:18 smithi150 ceph-mon[33096]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:18 smithi115 ceph-mon[28968]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:18 smithi115 ceph-mon[39520]: pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:20.461 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:20.770 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:20.771 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:20.771 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:20 smithi150 ceph-mon[33096]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:21.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:20 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/9022359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:21.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:20 smithi115 ceph-mon[39520]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:21.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:20 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/9022359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:21.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:20 smithi115 ceph-mon[28968]: pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:21.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:20 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/9022359' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:23.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:22 smithi150 ceph-mon[33096]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:23.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:22 smithi115 ceph-mon[28968]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:23.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:22 smithi115 ceph-mon[39520]: pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:24 smithi150 ceph-mon[33096]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:25.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:24 smithi115 ceph-mon[28968]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:24] "GET /metrics HTTP/1.1" 200 219787 "" "Prometheus/2.43.0" 2024-09-10T19:19:25.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:24 smithi115 ceph-mon[39520]: pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:25.773 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:26.083 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:26.084 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:26.084 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:26.084 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:25 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:26.084 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:25 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:25 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:26 smithi150 ceph-mon[33096]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:27.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:26 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/310878449' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:26 smithi115 ceph-mon[28968]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:26 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/310878449' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:27.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:26 smithi115 ceph-mon[39520]: pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:27.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:26 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/310878449' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:29.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:28 smithi150 ceph-mon[33096]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:29.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:29.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:28 smithi115 ceph-mon[39520]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:29.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:28 smithi115 ceph-mon[28968]: pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:31.085 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:31.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:30 smithi150 ceph-mon[33096]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:31.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:30 smithi115 ceph-mon[28968]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:31.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:30 smithi115 ceph-mon[39520]: pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:31.397 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:31.397 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:31.397 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:32.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:31 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1491827004' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:32.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:31 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1491827004' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:32.213 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:31 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1491827004' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:33.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:32 smithi150 ceph-mon[33096]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:33.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:32 smithi115 ceph-mon[28968]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:33.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:32 smithi115 ceph-mon[39520]: pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:34.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:33 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:34.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:33 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:34.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:33 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:35.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:34 smithi150 ceph-mon[33096]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:35.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:35.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:34] "GET /metrics HTTP/1.1" 200 219781 "" "Prometheus/2.43.0" 2024-09-10T19:19:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:34 smithi115 ceph-mon[28968]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:35.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:34 smithi115 ceph-mon[39520]: pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:36.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:35 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:36.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:35 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:36.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:35 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:36.399 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:36.708 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:36.709 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:36.709 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:36 smithi115 ceph-mon[28968]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:37.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:36 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2831817752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:36 smithi115 ceph-mon[39520]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:37.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:36 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2831817752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:37.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:36 smithi150 ceph-mon[33096]: pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:37.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:36 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2831817752' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:39.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:38 smithi150 ceph-mon[33096]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:39.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:39.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:38 smithi115 ceph-mon[28968]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:39.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:38 smithi115 ceph-mon[39520]: pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:41.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:40 smithi115 ceph-mon[28968]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:41.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:40 smithi115 ceph-mon[39520]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:40 smithi150 ceph-mon[33096]: pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:41.710 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:42.024 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:43.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:42 smithi115 ceph-mon[28968]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:43.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:42 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1494063783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:43.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:42 smithi115 ceph-mon[39520]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:43.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:42 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1494063783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:43.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:42 smithi150 ceph-mon[33096]: pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:43.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:42 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1494063783' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:44 smithi150 ceph-mon[33096]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:45.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:44 smithi115 ceph-mon[28968]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:45.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:44] "GET /metrics HTTP/1.1" 200 219781 "" "Prometheus/2.43.0" 2024-09-10T19:19:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:44 smithi115 ceph-mon[39520]: pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:46.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:45 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:45 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:46.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:45 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:47.025 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:47.336 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:47.337 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:47.337 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:47.337 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:46 smithi115 ceph-mon[28968]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:47.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:46 smithi115 ceph-mon[39520]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:47.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:46 smithi150 ceph-mon[33096]: pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:48.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:47 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3726004442' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:48.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:47 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3726004442' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:48.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:47 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3726004442' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:48.923 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:48 smithi150 ceph-mon[33096]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:48 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:49.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:48 smithi115 ceph-mon[28968]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:49.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:48 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:49.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:48 smithi115 ceph-mon[39520]: pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:49.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:48 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:19:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:50 smithi150 ceph-mon[33096]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:50 smithi115 ceph-mon[28968]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:51.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:50 smithi115 ceph-mon[39520]: pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:52.338 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:52.648 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:52.648 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:52.648 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:53.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:52 smithi150 ceph-mon[33096]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:53.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:52 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3696348296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:53.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:52 smithi115 ceph-mon[39520]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:53.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:52 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3696348296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:53.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:52 smithi115 ceph-mon[28968]: pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:53.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:52 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3696348296' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:55.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:54 smithi150 ceph-mon[33096]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:55.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:19:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:19:55.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:54 smithi115 ceph-mon[28968]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:55.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:19:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:54] "GET /metrics HTTP/1.1" 200 219745 "" "Prometheus/2.43.0" 2024-09-10T19:19:55.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:54 smithi115 ceph-mon[39520]: pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:56.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:56.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:56.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:19:57.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:56 smithi150 ceph-mon[33096]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:57.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:56 smithi115 ceph-mon[39520]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:57.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:56 smithi115 ceph-mon[28968]: pgmap v106: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:19:57.650 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:19:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:19:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:19:57.962 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:19:58.939 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:19:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:19:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:19:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:58 smithi150 ceph-mon[33096]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:19:58 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2589539341' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:59.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:58 smithi115 ceph-mon[28968]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:59.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:19:58 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2589539341' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:19:59.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:58 smithi115 ceph-mon[39520]: pgmap v107: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:19:59.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:19:58 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2589539341' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:01.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:00 smithi150 ceph-mon[33096]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:01.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:00 smithi150 ceph-mon[33096]: overall HEALTH_OK 2024-09-10T19:20:01.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:00 smithi115 ceph-mon[28968]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:01.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:00 smithi115 ceph-mon[28968]: overall HEALTH_OK 2024-09-10T19:20:01.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:00 smithi115 ceph-mon[39520]: pgmap v108: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:01.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:00 smithi115 ceph-mon[39520]: overall HEALTH_OK 2024-09-10T19:20:02.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:01 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:20:02.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:01 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:20:02.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:01 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:20:02.964 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:03.276 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:03.276 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:03.276 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:03.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[28968]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:03.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:20:03.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:20:03.276 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:20:03.277 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[39520]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:03.277 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:20:03.277 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:20:03.277 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:20:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:02 smithi150 ceph-mon[33096]: pgmap v109: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:20:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:20:03.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:20:04.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:03 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1911278349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:04.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:04.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:03 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1911278349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:04.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:04.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:03 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1911278349' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:04.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:05.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:05.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:04 smithi150 ceph-mon[33096]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:05.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:04 smithi115 ceph-mon[28968]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:05.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:04] "GET /metrics HTTP/1.1" 200 219744 "" "Prometheus/2.43.0" 2024-09-10T19:20:05.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:04 smithi115 ceph-mon[39520]: pgmap v110: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:06.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:06.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:06.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:06 smithi150 ceph-mon[33096]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:07.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:06 smithi115 ceph-mon[28968]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:07.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:06 smithi115 ceph-mon[39520]: pgmap v111: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:08.277 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:08.591 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:08.969 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:09.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:08 smithi150 ceph-mon[33096]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:09.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:08 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3062528021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:09.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:08 smithi115 ceph-mon[28968]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:09.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:08 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3062528021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:09.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:08 smithi115 ceph-mon[39520]: pgmap v112: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:09.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:08 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3062528021' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:11.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:10 smithi150 ceph-mon[33096]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:11.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:10 smithi115 ceph-mon[28968]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:11.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:10 smithi115 ceph-mon[39520]: pgmap v113: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:13.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:12 smithi150 ceph-mon[33096]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:13.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:12 smithi115 ceph-mon[28968]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:13.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:12 smithi115 ceph-mon[39520]: pgmap v114: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:13.593 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:13.908 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:13.908 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:13.908 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:13 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2296330414' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:13 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2296330414' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:14.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:13 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2296330414' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:14 smithi150 ceph-mon[33096]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:15.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:15.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:14] "GET /metrics HTTP/1.1" 200 219744 "" "Prometheus/2.43.0" 2024-09-10T19:20:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:14 smithi115 ceph-mon[28968]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:15.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:14 smithi115 ceph-mon[39520]: pgmap v115: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:16.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:15 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:15 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:16.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:15 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:17.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:16 smithi150 ceph-mon[33096]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:17.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:16 smithi115 ceph-mon[28968]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:17.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:16 smithi115 ceph-mon[39520]: pgmap v116: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:18.911 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:18.999 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:18 smithi115 ceph-mon[28968]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:18 smithi115 ceph-mon[39520]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:19.228 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:19.228 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:19.228 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:18 smithi150 ceph-mon[33096]: pgmap v117: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:19.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:20.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:20 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1185967095' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:20 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1185967095' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:20.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:20 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1185967095' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:21.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:21 smithi150 ceph-mon[33096]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:21 smithi115 ceph-mon[28968]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:21.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:21 smithi115 ceph-mon[39520]: pgmap v118: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:23.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:23 smithi150 ceph-mon[33096]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:23 smithi115 ceph-mon[28968]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:23.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:23 smithi115 ceph-mon[39520]: pgmap v119: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:24.230 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:24.542 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:25.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:25 smithi150 ceph-mon[33096]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:25 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3080485642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:25.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:24] "GET /metrics HTTP/1.1" 200 219745 "" "Prometheus/2.43.0" 2024-09-10T19:20:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:25 smithi115 ceph-mon[28968]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:25 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3080485642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:25.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:25 smithi115 ceph-mon[39520]: pgmap v120: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:25.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:25 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3080485642' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:26.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:26 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:26.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:26 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:26.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:26 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:27.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:27 smithi150 ceph-mon[33096]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:27 smithi115 ceph-mon[28968]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:27.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:27 smithi115 ceph-mon[39520]: pgmap v121: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:29.021 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:29.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:29 smithi150 ceph-mon[33096]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:29.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:29 smithi115 ceph-mon[28968]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:29.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:29 smithi115 ceph-mon[39520]: pgmap v122: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:29.544 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:29.855 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:30.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:30 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3733392413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:30.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:30 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3733392413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:30.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:30 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3733392413' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:31.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:31 smithi150 ceph-mon[33096]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:31 smithi115 ceph-mon[28968]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:31.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:31 smithi115 ceph-mon[39520]: pgmap v123: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:33.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:33 smithi150 ceph-mon[33096]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:33 smithi115 ceph-mon[28968]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:33.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:33 smithi115 ceph-mon[39520]: pgmap v124: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:34.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:34 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:34.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:34 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:34.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:34 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:34.857 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:35.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:35 smithi115 ceph-mon[28968]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:35.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:35 smithi115 ceph-mon[39520]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:35.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:34] "GET /metrics HTTP/1.1" 200 219760 "" "Prometheus/2.43.0" 2024-09-10T19:20:35.169 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:35.170 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:35.170 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:35.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:35 smithi150 ceph-mon[33096]: pgmap v125: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:35.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:36.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:36 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:36.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:36 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3670462322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:36.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:36 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:36.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:36 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3670462322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:36.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:36 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:36.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:36 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3670462322' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:37.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:37 smithi150 ceph-mon[33096]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:37.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:37 smithi115 ceph-mon[39520]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:37 smithi115 ceph-mon[28968]: pgmap v126: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:39.043 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:39.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:39 smithi150 ceph-mon[33096]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:39.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:39 smithi115 ceph-mon[39520]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:39 smithi115 ceph-mon[28968]: pgmap v127: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:40.171 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:40.484 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:40.484 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:40.484 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:41 smithi150 ceph-mon[33096]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:41.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:41 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3016089433' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:41 smithi115 ceph-mon[28968]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:41.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:41 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3016089433' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:41.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:41 smithi115 ceph-mon[39520]: pgmap v128: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:41.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:41 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3016089433' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:43.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:43 smithi150 ceph-mon[33096]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:43.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:43 smithi115 ceph-mon[28968]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:43.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:43 smithi115 ceph-mon[39520]: pgmap v129: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:45 smithi115 ceph-mon[28968]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:45.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:44] "GET /metrics HTTP/1.1" 200 219760 "" "Prometheus/2.43.0" 2024-09-10T19:20:45.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:45 smithi115 ceph-mon[39520]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:45.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:45 smithi150 ceph-mon[33096]: pgmap v130: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:45.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:45.486 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:45.803 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:45.803 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:45.803 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:46.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:46 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:46.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:46 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3972932439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:46.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:46 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:46.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:46 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3972932439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:46 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:46.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:46 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3972932439' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:47.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:47 smithi150 ceph-mon[33096]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:47 smithi115 ceph-mon[28968]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:47.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:47 smithi115 ceph-mon[39520]: pgmap v131: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:49.061 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:49 smithi150 ceph-mon[33096]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:49.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:49 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:49.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:49 smithi115 ceph-mon[28968]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:49.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:49 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:49.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:49 smithi115 ceph-mon[39520]: pgmap v132: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:49.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:49 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:20:50.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:50 smithi150 ceph-mon[33096]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:50.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:50 smithi115 ceph-mon[28968]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:50.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:50 smithi115 ceph-mon[39520]: pgmap v133: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:50.804 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:51.117 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:51.117 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:51.118 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:51.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:51 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2199051523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:51.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:51 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2199051523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:51.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:51 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2199051523' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:52.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:52 smithi150 ceph-mon[33096]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:52.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:52 smithi115 ceph-mon[28968]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:52.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:52 smithi115 ceph-mon[39520]: pgmap v134: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:55.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:54 smithi150 ceph-mon[33096]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:55.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:20:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:20:55.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:54 smithi115 ceph-mon[39520]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:55.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:54 smithi115 ceph-mon[28968]: pgmap v135: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:55.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:20:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:54] "GET /metrics HTTP/1.1" 200 219755 "" "Prometheus/2.43.0" 2024-09-10T19:20:56.119 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:20:56.120 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:56.120 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:56.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:20:56.431 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:20:56.432 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:20:56.432 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:20:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:56 smithi150 ceph-mon[33096]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:56 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2267679253' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:56 smithi115 ceph-mon[28968]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:56 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2267679253' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:56 smithi115 ceph-mon[39520]: pgmap v136: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:20:57.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:56 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2267679253' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:20:59.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:20:58 smithi150 ceph-mon[33096]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:59.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:20:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:20:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:20:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:20:58 smithi115 ceph-mon[28968]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:20:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:20:58 smithi115 ceph-mon[39520]: pgmap v137: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:01.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:00 smithi150 ceph-mon[33096]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:00 smithi115 ceph-mon[28968]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:01.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:00 smithi115 ceph-mon[39520]: pgmap v138: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:01.433 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:21:01.745 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:21:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:21:01.746 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3078558516' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:02.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3078558516' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3078558516' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:03.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:02 smithi150 ceph-mon[33096]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:03.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:03.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[28968]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[39520]: pgmap v139: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:03.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:04.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:04.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:05.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:04 smithi150 ceph-mon[33096]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:05.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:05.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:04] "GET /metrics HTTP/1.1" 200 219767 "" "Prometheus/2.43.0" 2024-09-10T19:21:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:04 smithi115 ceph-mon[28968]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:05.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:04 smithi115 ceph-mon[39520]: pgmap v140: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:06.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:06.747 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:21:07.065 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:21:07.066 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:21:07.066 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:07.066 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:06 smithi115 ceph-mon[39520]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:07.066 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:06 smithi115 ceph-mon[28968]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:07.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:06 smithi150 ceph-mon[33096]: pgmap v141: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:08.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:07 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1924965395' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:08.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:07 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1924965395' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:08.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:07 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1924965395' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:09.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:08 smithi150 ceph-mon[33096]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:09.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:08 smithi115 ceph-mon[28968]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:09.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:08 smithi115 ceph-mon[39520]: pgmap v142: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:11.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:10 smithi150 ceph-mon[33096]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:11.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:10 smithi115 ceph-mon[28968]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:11.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:10 smithi115 ceph-mon[39520]: pgmap v143: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:12.067 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:21:12.377 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== 2024-09-10T19:21:12.377 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== ']' 2024-09-10T19:21:12.378 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:12 smithi150 ceph-mon[33096]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/835265966' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:12 smithi115 ceph-mon[28968]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/835265966' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:12 smithi115 ceph-mon[39520]: pgmap v144: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/835265966' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:14 smithi150 ceph-mon[33096]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:15.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:14 smithi115 ceph-mon[28968]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:15.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:14] "GET /metrics HTTP/1.1" 200 219767 "" "Prometheus/2.43.0" 2024-09-10T19:21:15.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:14 smithi115 ceph-mon[39520]: pgmap v145: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:21:16.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:15 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:16.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:15 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:16.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:15 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:17.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:16 smithi115 ceph-mon[28968]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:17.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:16 smithi115 ceph-mon[39520]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:17.379 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.0 2024-09-10T19:21:17.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:16 smithi150 ceph-mon[33096]: pgmap v146: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:17.695 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.1 2024-09-10T19:21:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBSm+Bmxw5QIxAA+6w1I1HU5b7CB6umMuA8sw== 2024-09-10T19:21:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAumuBm9bD5FRAA52nI4TWYYMx484ccUd7hnQ== == AQBSm+Bmxw5QIxAA+6w1I1HU5b7CB6umMuA8sw== ']' 2024-09-10T19:21:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:21:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.1' 2024-09-10T19:21:17.696 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:17.997 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.1 2024-09-10T19:21:17.997 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:17 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3846662220' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:17.997 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:17 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3846662220' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:18.252 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.1 on host 'smithi115' 2024-09-10T19:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== ']' 2024-09-10T19:21:18.264 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:18.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:17 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3846662220' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.0"}]: dispatch 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1277377849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:19.026 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1277377849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.027 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:21:19.028 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:19.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: pgmap v147: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1277377849' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]: dispatch 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.1", "format": "json"}]': finished 2024-09-10T19:21:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:20.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: from='client.15036 v1:172.21.15.115:0/1021529447' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.1 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: Rotating authentication key for osd.1 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: from='client.15036 v1:172.21.15.115:0/1021529447' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.1 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: Rotating authentication key for osd.1 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T19:21:20.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:20.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:19 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:20.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: from='client.15036 v1:172.21.15.115:0/1021529447' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:20.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.1 2024-09-10T19:21:20.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: Rotating authentication key for osd.1 2024-09-10T19:21:20.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.1 on smithi115 2024-09-10T19:21:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:20.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:19 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:21 smithi150 ceph-mon[33096]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:21.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:21 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:21 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[28968]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[39520]: pgmap v148: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:21.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:21 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:23.268 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:23.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:23 smithi150 ceph-mon[33096]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:23.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:23 smithi115 ceph-mon[28968]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:23.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:23 smithi115 ceph-mon[39520]: pgmap v149: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:23.580 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:23.580 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== ']' 2024-09-10T19:21:23.580 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:24.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:24 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3078755120' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:24.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:24 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3078755120' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:24.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:24 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3078755120' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:25 smithi115 ceph-mon[28968]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:24] "GET /metrics HTTP/1.1" 200 219759 "" "Prometheus/2.43.0" 2024-09-10T19:21:25.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:25 smithi115 ceph-mon[39520]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:25.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:25 smithi150 ceph-mon[33096]: pgmap v150: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:25.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:26.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:26 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:26.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:26 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:26.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:26 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:27.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:27 smithi150 ceph-mon[33096]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:27.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:27 smithi115 ceph-mon[28968]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:27.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:27 smithi115 ceph-mon[39520]: pgmap v151: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:28.582 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:28.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== ']' 2024-09-10T19:21:28.894 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:29.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:29 smithi115 ceph-mon[28968]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:29.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:29 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1979951370' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:29.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:29 smithi115 ceph-mon[39520]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:29.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:29 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1979951370' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:29.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:29 smithi150 ceph-mon[33096]: pgmap v152: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:29.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:29 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1979951370' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:31.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:31 smithi115 ceph-mon[39520]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:31 smithi115 ceph-mon[28968]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:31.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:31 smithi150 ceph-mon[33096]: pgmap v153: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:33.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:33 smithi115 ceph-mon[39520]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:33.417 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:33 smithi115 ceph-mon[28968]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:33.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:33 smithi150 ceph-mon[33096]: pgmap v154: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:33.896 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:34.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:34 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:34.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:34 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:34.205 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:34.205 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== ']' 2024-09-10T19:21:34.205 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:34.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:34 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:35.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:34] "GET /metrics HTTP/1.1" 200 219761 "" "Prometheus/2.43.0" 2024-09-10T19:21:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:35 smithi115 ceph-mon[28968]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:35.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:35 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3492262657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:35 smithi115 ceph-mon[39520]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:35.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:35 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3492262657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:35.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:35.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:35 smithi150 ceph-mon[33096]: pgmap v155: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:35.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:35 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3492262657' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:36.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:36 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:36.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:36 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:36.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:36 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:37 smithi115 ceph-mon[28968]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:37.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:37 smithi115 ceph-mon[39520]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:37 smithi150 ceph-mon[33096]: pgmap v156: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:38.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:39.207 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:39.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:39 smithi115 ceph-mon[28968]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:39.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:39 smithi115 ceph-mon[39520]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:39.519 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== 2024-09-10T19:21:39.520 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== ']' 2024-09-10T19:21:39.520 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:39.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:39 smithi150 ceph-mon[33096]: pgmap v157: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:40.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/733824246' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:40.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/733824246' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:40.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/733824246' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:41 smithi150 ceph-mon[33096]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:41 smithi115 ceph-mon[28968]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:41.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:41 smithi115 ceph-mon[39520]: pgmap v158: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:43.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:43 smithi150 ceph-mon[33096]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:43.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:43 smithi115 ceph-mon[28968]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:43.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:43 smithi115 ceph-mon[39520]: pgmap v159: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:44.521 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.1 2024-09-10T19:21:44.837 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.2 2024-09-10T19:21:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAunOBms2DqExAA8ZWjxT3CBis0C5V1F4zraw== 2024-09-10T19:21:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBFmuBmHAmQABAAqVQqAE1uVfOZ0nj68Nf66A== == AQAunOBms2DqExAA8ZWjxT3CBis0C5V1F4zraw== ']' 2024-09-10T19:21:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:21:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.2' 2024-09-10T19:21:44.838 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T19:21:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== 2024-09-10T19:21:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== 2024-09-10T19:21:45.139 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.2 2024-09-10T19:21:45.140 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:44] "GET /metrics HTTP/1.1" 200 219761 "" "Prometheus/2.43.0" 2024-09-10T19:21:45.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:44 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:45 smithi150 ceph-mon[33096]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/940759024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1252323364' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:45.393 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.2 on host 'smithi115' 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[39520]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/940759024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1252323364' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[28968]: pgmap v160: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/940759024' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.1"}]: dispatch 2024-09-10T19:21:45.394 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1252323364' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:45.404 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== == AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== ']' 2024-09-10T19:21:45.405 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:46.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:46.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:46.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:46 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:46.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]: dispatch 2024-09-10T19:21:46.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:46 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.2", "format": "json"}]': finished 2024-09-10T19:21:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[28968]: from='client.15090 v1:172.21.15.115:0/4208419097' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.2 2024-09-10T19:21:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[28968]: Rotating authentication key for osd.2 2024-09-10T19:21:47.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[28968]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[39520]: from='client.15090 v1:172.21.15.115:0/4208419097' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.2 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[39520]: Rotating authentication key for osd.2 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T19:21:47.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:47 smithi115 ceph-mon[39520]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:47 smithi150 ceph-mon[33096]: from='client.15090 v1:172.21.15.115:0/4208419097' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:21:47.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:47 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.2 2024-09-10T19:21:47.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:47 smithi150 ceph-mon[33096]: Rotating authentication key for osd.2 2024-09-10T19:21:47.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:47 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.2 on smithi115 2024-09-10T19:21:47.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:47 smithi150 ceph-mon[33096]: pgmap v161: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.471 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.472 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:48 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:48 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:48 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:48 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:48.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:48 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:21:49.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:49.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:49 smithi150 ceph-mon[33096]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:49.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:49 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:49 smithi115 ceph-mon[28968]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:49 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:49.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:49 smithi115 ceph-mon[39520]: pgmap v162: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:49.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:49 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:21:50.408 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T19:21:50.721 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== 2024-09-10T19:21:50.721 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== == AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== ']' 2024-09-10T19:21:50.721 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:51.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:51 smithi150 ceph-mon[33096]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:51.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:51 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1771960917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:51 smithi115 ceph-mon[28968]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:51 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1771960917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:51.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:51 smithi115 ceph-mon[39520]: pgmap v163: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:51.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:51 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1771960917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:52.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:52 smithi150 ceph-mon[33096]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:52.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:52 smithi115 ceph-mon[28968]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:52.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:52 smithi115 ceph-mon[39520]: pgmap v164: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:55.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:54 smithi150 ceph-mon[33096]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:55.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:21:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:21:55.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:54 smithi115 ceph-mon[28968]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:55.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:21:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:54] "GET /metrics HTTP/1.1" 200 219758 "" "Prometheus/2.43.0" 2024-09-10T19:21:55.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:54 smithi115 ceph-mon[39520]: pgmap v165: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:55.723 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T19:21:56.037 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== 2024-09-10T19:21:56.038 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== == AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== ']' 2024-09-10T19:21:56.038 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:21:56.038 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:56.038 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:56.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:21:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:56 smithi150 ceph-mon[33096]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:57.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:56 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/13890036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:56 smithi115 ceph-mon[39520]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:56 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/13890036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:56 smithi115 ceph-mon[28968]: pgmap v166: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:21:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:56 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/13890036' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:21:59.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:21:58 smithi150 ceph-mon[33096]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:59.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:21:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:21:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:21:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:21:58 smithi115 ceph-mon[39520]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:21:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:21:58 smithi115 ceph-mon[28968]: pgmap v167: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:01.039 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T19:22:01.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:00 smithi150 ceph-mon[33096]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:01.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:00 smithi115 ceph-mon[39520]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:01.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:00 smithi115 ceph-mon[28968]: pgmap v168: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:01.354 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== 2024-09-10T19:22:01.355 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== == AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== ']' 2024-09-10T19:22:01.355 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:02.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/499304727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:02.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/499304727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:02.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/499304727' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:03.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:02 smithi115 ceph-mon[28968]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:03.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:02 smithi115 ceph-mon[39520]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:03.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:02 smithi150 ceph-mon[33096]: pgmap v169: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:04.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:04.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:05.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:04 smithi115 ceph-mon[39520]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:04 smithi115 ceph-mon[28968]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:05.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:04] "GET /metrics HTTP/1.1" 200 219776 "" "Prometheus/2.43.0" 2024-09-10T19:22:05.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:04 smithi150 ceph-mon[33096]: pgmap v170: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:05.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:06.357 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.2 2024-09-10T19:22:06.399 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.3 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBJnOBmVfbJLBAAe/m7BxiBXLRnkd5FOlvj9g== 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBbmuBmVdEXDRAAJFXt6SiyJ4FofF8d5tSWNg== == AQBJnOBmVfbJLBAAe/m7BxiBXLRnkd5FOlvj9g== ']' 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.3' 2024-09-10T19:22:06.667 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T19:22:06.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== 2024-09-10T19:22:06.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== 2024-09-10T19:22:06.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.3 2024-09-10T19:22:06.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:06 smithi115 ceph-mon[28968]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:06.971 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:06 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2028525129' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:06.971 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:06 smithi115 ceph-mon[39520]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:06.971 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:06 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2028525129' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:07.225 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.3 on host 'smithi115' 2024-09-10T19:22:07.236 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== == AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== ']' 2024-09-10T19:22:07.236 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:06 smithi150 ceph-mon[33096]: pgmap v171: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:07.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:06 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2028525129' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.2"}]: dispatch 2024-09-10T19:22:08.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3244777375' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:08.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='client.15138 v1:172.21.15.115:0/474961478' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:08.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.3 2024-09-10T19:22:08.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3244777375' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:08.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='client.15138 v1:172.21.15.115:0/474961478' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.3 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:08.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:07 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3244777375' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='client.15138 v1:172.21.15.115:0/474961478' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.3 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:08.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:07 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:09.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: Rotating authentication key for osd.3 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T19:22:09.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: Rotating authentication key for osd.3 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:22:09.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: pgmap v172: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: Rotating authentication key for osd.3 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]: dispatch 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.3", "format": "json"}]': finished 2024-09-10T19:22:09.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:08 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.3 on smithi115 2024-09-10T19:22:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:10 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:10 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:10 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:10 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:10 smithi150 ceph-mon[33096]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[28968]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:10.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:10.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:10 smithi115 ceph-mon[39520]: pgmap v173: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:12.239 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T19:22:12.553 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== 2024-09-10T19:22:12.553 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== == AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== ']' 2024-09-10T19:22:12.553 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:13.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:12 smithi150 ceph-mon[33096]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3778392424' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:12 smithi115 ceph-mon[28968]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3778392424' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:12 smithi115 ceph-mon[39520]: pgmap v174: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:13.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3778392424' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:15.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:14 smithi150 ceph-mon[33096]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:15.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:15.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:14 smithi115 ceph-mon[28968]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:15.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:14] "GET /metrics HTTP/1.1" 200 219776 "" "Prometheus/2.43.0" 2024-09-10T19:22:15.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:14 smithi115 ceph-mon[39520]: pgmap v175: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:16.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:15 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:16.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:15 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:16.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:15 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:17.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:16 smithi150 ceph-mon[33096]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:17.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:16 smithi115 ceph-mon[39520]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:17.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:16 smithi115 ceph-mon[28968]: pgmap v176: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:17.555 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T19:22:17.860 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== 2024-09-10T19:22:17.860 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== == AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== ']' 2024-09-10T19:22:17.860 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:18.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:17 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2785357608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:18.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:17 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2785357608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:18.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:17 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2785357608' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:19.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:18 smithi150 ceph-mon[33096]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:19.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:18 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:19.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:18 smithi115 ceph-mon[28968]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:18 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:18 smithi115 ceph-mon[39520]: pgmap v177: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:19.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:18 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:21.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:20 smithi150 ceph-mon[33096]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:21.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:20 smithi115 ceph-mon[28968]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:21.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:20 smithi115 ceph-mon[39520]: pgmap v178: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:22.861 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T19:22:23.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:22 smithi150 ceph-mon[33096]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:23.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:22 smithi115 ceph-mon[28968]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:23.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:22 smithi115 ceph-mon[39520]: pgmap v179: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== 2024-09-10T19:22:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== == AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== ']' 2024-09-10T19:22:23.169 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:24.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:23 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4090427289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:24.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:23 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4090427289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:24.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:23 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4090427289' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:25.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:24 smithi150 ceph-mon[33096]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:25.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:24] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T19:22:25.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:24 smithi115 ceph-mon[28968]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:25.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:24 smithi115 ceph-mon[39520]: pgmap v180: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:26.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:25 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:26.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:25 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:26.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:25 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:27.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:26 smithi150 ceph-mon[33096]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:27.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:26 smithi115 ceph-mon[28968]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:27.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:26 smithi115 ceph-mon[39520]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:28.171 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.3 2024-09-10T19:22:28.491 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.4 2024-09-10T19:22:28.492 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQBfnOBmZPNZORAAvABBq6lctZ4dTHT5TXHowg== 2024-09-10T19:22:28.492 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQBymuBmIhciCBAAvXaNfbwNl0h96AaYd3dv2Q== == AQBfnOBmZPNZORAAvABBq6lctZ4dTHT5TXHowg== ']' 2024-09-10T19:22:28.492 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:22:28.492 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.4' 2024-09-10T19:22:28.492 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T19:22:28.790 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== 2024-09-10T19:22:28.791 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== 2024-09-10T19:22:28.791 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.4 2024-09-10T19:22:29.049 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.4 on host 'smithi150' 2024-09-10T19:22:29.049 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[28968]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:29.049 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2344401088' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:29.049 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2610437641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:29.049 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[39520]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:29.050 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2344401088' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:29.050 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:28 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2610437641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:29.060 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== == AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== ']' 2024-09-10T19:22:29.061 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:29.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:28 smithi150 ceph-mon[33096]: pgmap v182: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:28 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2344401088' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.3"}]: dispatch 2024-09-10T19:22:29.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:28 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2610437641' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='client.15186 v1:172.21.15.115:0/1782879896' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.4 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:30.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: Rotating authentication key for osd.4 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:22:30.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:30 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T19:22:30.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='client.15186 v1:172.21.15.115:0/1782879896' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.4 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: Rotating authentication key for osd.4 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:22:30.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='client.15186 v1:172.21.15.115:0/1782879896' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.4 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: Rotating authentication key for osd.4 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]: dispatch 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.4", "format": "json"}]': finished 2024-09-10T19:22:30.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:30 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.4 on smithi150 2024-09-10T19:22:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[28968]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[39520]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:31 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:31 smithi150 ceph-mon[33096]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:31 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:31 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:31 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:31.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:31 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:33.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:33 smithi115 ceph-mon[28968]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:33.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:33 smithi115 ceph-mon[39520]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:33.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:33 smithi150 ceph-mon[33096]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:34.062 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T19:22:34.376 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== 2024-09-10T19:22:34.376 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== == AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== ']' 2024-09-10T19:22:34.376 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:34.377 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:34 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:34.377 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:34 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:34.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:34 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:35.153 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:34] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T19:22:35.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:35 smithi150 ceph-mon[33096]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:35.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:35 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2417989683' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:35.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:34 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:35 smithi115 ceph-mon[28968]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:35.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:35 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2417989683' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:35.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:35 smithi115 ceph-mon[39520]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:35.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:35 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2417989683' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:36.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:36 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:36.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:36 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:36.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:36 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:37.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:37 smithi115 ceph-mon[28968]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:37.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:37 smithi115 ceph-mon[39520]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:37.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:37 smithi150 ceph-mon[33096]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:39.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:39.378 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T19:22:39.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:39 smithi150 ceph-mon[33096]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:39.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:39 smithi115 ceph-mon[28968]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:39.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:39 smithi115 ceph-mon[39520]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:39.687 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== 2024-09-10T19:22:39.687 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== == AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== ']' 2024-09-10T19:22:39.688 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:40.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1101056901' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:40.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1101056901' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:40.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1101056901' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:41 smithi150 ceph-mon[33096]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:41 smithi115 ceph-mon[28968]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:41.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:41 smithi115 ceph-mon[39520]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:43.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:43 smithi150 ceph-mon[33096]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:43.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:43 smithi115 ceph-mon[28968]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:43.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:43 smithi115 ceph-mon[39520]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:44.689 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T19:22:45.005 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== 2024-09-10T19:22:45.005 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== == AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== ']' 2024-09-10T19:22:45.006 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:45.006 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:44] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T19:22:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:45 smithi150 ceph-mon[33096]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:45.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4190655789' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:45.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:45 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:45.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:45 smithi115 ceph-mon[28968]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:45.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4190655789' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:45.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:45 smithi115 ceph-mon[39520]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:45.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4190655789' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:46.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:46 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:46.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:46 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:46.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:46 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:47.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:47 smithi150 ceph-mon[33096]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:47.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:47 smithi115 ceph-mon[28968]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:47.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:47 smithi115 ceph-mon[39520]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:49.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:49.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:49 smithi150 ceph-mon[33096]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:49.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:49 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:49 smithi115 ceph-mon[28968]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:49.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:49 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:49.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:49 smithi115 ceph-mon[39520]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:49.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:49 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:22:50.007 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.4 2024-09-10T19:22:50.260 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:50 smithi115 ceph-mon[28968]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:50.325 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.5 2024-09-10T19:22:50.325 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQB1nOBmjbz9BxAA7DZbdJvfCQPhKLTiqC4DVg== 2024-09-10T19:22:50.326 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCHmuBmPSfpKxAAvEFVCiTCRUIvGP5itp2FWA== == AQB1nOBmjbz9BxAA7DZbdJvfCQPhKLTiqC4DVg== ']' 2024-09-10T19:22:50.326 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:22:50.326 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.5' 2024-09-10T19:22:50.326 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T19:22:50.625 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== 2024-09-10T19:22:50.625 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== 2024-09-10T19:22:50.625 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.5 2024-09-10T19:22:50.625 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:50 smithi115 ceph-mon[39520]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:50.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:50 smithi150 ceph-mon[33096]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:50.878 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.5 on host 'smithi150' 2024-09-10T19:22:50.889 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== == AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== ']' 2024-09-10T19:22:50.889 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2939798862' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4142135917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='client.15234 v1:172.21.15.115:0/3367927981' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.5 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: Rotating authentication key for osd.5 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.567 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:22:51.568 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:51 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2939798862' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4142135917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='client.15234 v1:172.21.15.115:0/3367927981' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.5 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: Rotating authentication key for osd.5 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T19:22:51.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2939798862' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.4"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4142135917' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='client.15234 v1:172.21.15.115:0/3367927981' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.5 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: Rotating authentication key for osd.5 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]: dispatch 2024-09-10T19:22:51.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.5", "format": "json"}]': finished 2024-09-10T19:22:51.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:51 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.5 on smithi150 2024-09-10T19:22:52.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:52 smithi150 ceph-mon[33096]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:52.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:52 smithi115 ceph-mon[28968]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:52.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:52 smithi115 ceph-mon[39520]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.883 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:53 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:53.914 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:53 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:22:54.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:54 smithi150 ceph-mon[33096]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:54.902 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:54 smithi115 ceph-mon[28968]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:54.902 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:54 smithi115 ceph-mon[39520]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:55.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:22:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:54] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T19:22:55.382 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:22:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:22:55.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:55.891 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T19:22:55.891 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:55.892 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:22:56.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== 2024-09-10T19:22:56.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== == AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== ']' 2024-09-10T19:22:56.203 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:22:56.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:56 smithi150 ceph-mon[33096]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:56.784 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:56 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2698345717' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:56.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:56 smithi115 ceph-mon[28968]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:56.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:56 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2698345717' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:56.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:56 smithi115 ceph-mon[39520]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:22:56.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:56 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2698345717' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:22:59.002 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:22:58 smithi150 ceph-mon[33096]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:59.003 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:22:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:22:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:22:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:22:58 smithi115 ceph-mon[28968]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:22:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:22:58 smithi115 ceph-mon[39520]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:00 smithi150 ceph-mon[33096]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:00.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:00 smithi115 ceph-mon[28968]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:00.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:00 smithi115 ceph-mon[39520]: pgmap v198: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:01.205 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T19:23:01.515 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== 2024-09-10T19:23:01.515 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== == AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== ']' 2024-09-10T19:23:01.516 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:01.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2477024248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:01.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2477024248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:01.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2477024248' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:02.882 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:02 smithi150 ceph-mon[33096]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:02.913 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:02 smithi115 ceph-mon[28968]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:02.913 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:02 smithi115 ceph-mon[39520]: pgmap v199: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:04.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:03 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:04.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:03 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:04.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:03 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:05.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:04 smithi150 ceph-mon[33096]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:05.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:05.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:04] "GET /metrics HTTP/1.1" 200 219803 "" "Prometheus/2.43.0" 2024-09-10T19:23:05.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:04 smithi115 ceph-mon[28968]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:05.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:04 smithi115 ceph-mon[39520]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:06.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:06.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:06.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:06.517 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T19:23:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== 2024-09-10T19:23:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== == AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== ']' 2024-09-10T19:23:06.828 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:07.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:06 smithi150 ceph-mon[33096]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:07.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:06 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1184731932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:06 smithi115 ceph-mon[39520]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:07.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:06 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1184731932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:06 smithi115 ceph-mon[28968]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:07.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:06 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1184731932' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:08.867 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:09.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:08 smithi150 ceph-mon[33096]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:09.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:08 smithi115 ceph-mon[28968]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:09.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:08 smithi115 ceph-mon[39520]: pgmap v202: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:10.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:10 smithi150 ceph-mon[33096]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:10.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:10 smithi115 ceph-mon[28968]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:10.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:10 smithi115 ceph-mon[39520]: pgmap v203: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:11.830 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.5 2024-09-10T19:23:12.143 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.6 2024-09-10T19:23:12.143 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQCKnOBmQwIHORAAb4v/O5mUyl5trr63u9i+WA== 2024-09-10T19:23:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQCfmuBmr5OSIxAA6IWcDx46YKiUnJM/2jUqag== == AQCKnOBmQwIHORAAb4v/O5mUyl5trr63u9i+WA== ']' 2024-09-10T19:23:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:23:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.6' 2024-09-10T19:23:12.144 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:12.443 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:12.444 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:12.444 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.6 2024-09-10T19:23:12.697 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.6 on host 'smithi150' 2024-09-10T19:23:12.709 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== ']' 2024-09-10T19:23:12.709 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4278550092' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1883631620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:12 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4278550092' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1883631620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:13.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: pgmap v204: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4278550092' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.5"}]: dispatch 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1883631620' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:13.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:12 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='client.15282 v1:172.21.15.115:0/1653165704' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.6 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: Rotating authentication key for osd.6 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:23:14.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:13 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T19:23:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='client.15282 v1:172.21.15.115:0/1653165704' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.6 2024-09-10T19:23:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:14.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: Rotating authentication key for osd.6 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='client.15282 v1:172.21.15.115:0/1653165704' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.6", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.6 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: Rotating authentication key for osd.6 2024-09-10T19:23:14.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]: dispatch 2024-09-10T19:23:14.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.6", "format": "json"}]': finished 2024-09-10T19:23:14.165 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:13 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.6 on smithi150 2024-09-10T19:23:15.155 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:14] "GET /metrics HTTP/1.1" 200 219803 "" "Prometheus/2.43.0" 2024-09-10T19:23:15.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-mon[33096]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:15.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.384 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.384 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:15.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[28968]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[39520]: pgmap v205: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:15 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:16.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:16 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:16.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:16 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:16.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:16 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:17.582 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:17 smithi115 ceph-mon[28968]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:17.583 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:17 smithi115 ceph-mon[39520]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:17.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:17 smithi150 ceph-mon[33096]: pgmap v206: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:17.711 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== ']' 2024-09-10T19:23:18.021 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:18.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2783121332' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:18.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:18 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2783121332' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:18.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:18 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2783121332' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:19.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:18] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:19.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:19 smithi150 ceph-mon[33096]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:19.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:19 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:19.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:19 smithi115 ceph-mon[28968]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:19.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:19 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:19.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:19 smithi115 ceph-mon[39520]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:19.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:19 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:21.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:21 smithi150 ceph-mon[33096]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:21.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:21 smithi115 ceph-mon[28968]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:21.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:21 smithi115 ceph-mon[39520]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:23.023 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:23.336 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:23.336 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== ']' 2024-09-10T19:23:23.336 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:23.336 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:23 smithi115 ceph-mon[28968]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:23.337 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:23 smithi115 ceph-mon[39520]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:23.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:23 smithi150 ceph-mon[33096]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:24.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:24 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3755459633' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:24.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:24 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3755459633' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:24.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:24 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3755459633' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:25.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:24] "GET /metrics HTTP/1.1" 200 219806 "" "Prometheus/2.43.0" 2024-09-10T19:23:25.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:25 smithi150 ceph-mon[33096]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:25.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:25.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:25 smithi115 ceph-mon[39520]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:25.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:25 smithi115 ceph-mon[28968]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:26.550 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:26 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:26.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:26 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:26.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:26 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:27.505 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:27 smithi150 ceph-mon[33096]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:27.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:27 smithi115 ceph-mon[28968]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:27.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:27 smithi115 ceph-mon[39520]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:28.338 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:28.652 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:28.652 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== ']' 2024-09-10T19:23:28.652 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:29.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:28 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:28] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:29.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:29 smithi150 ceph-mon[33096]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:29.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:29 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2826668853' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:29.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:29 smithi115 ceph-mon[39520]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:29.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:29 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2826668853' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:29.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:29 smithi115 ceph-mon[28968]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:29.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:29 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2826668853' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:31.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:31 smithi150 ceph-mon[33096]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:31.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:31 smithi115 ceph-mon[39520]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:31.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:31 smithi115 ceph-mon[28968]: pgmap v213: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:33.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:33 smithi150 ceph-mon[33096]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:33.654 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:33.654 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:33 smithi115 ceph-mon[28968]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:33.655 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:33 smithi115 ceph-mon[39520]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:33.969 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== 2024-09-10T19:23:33.969 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== ']' 2024-09-10T19:23:33.969 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:34.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:34 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:34.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:34 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2510659103' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:34.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:34 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:34.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:34 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2510659103' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:34.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:34 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:34.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:34 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2510659103' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:35.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:34] "GET /metrics HTTP/1.1" 200 219801 "" "Prometheus/2.43.0" 2024-09-10T19:23:35.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:35 smithi150 ceph-mon[33096]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:35.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:35 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:35.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:35 smithi115 ceph-mon[39520]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:35.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:35 smithi115 ceph-mon[28968]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:36.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:36 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:36.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:36 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:36.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:36 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:37.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:37 smithi150 ceph-mon[33096]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:37.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:37 smithi115 ceph-mon[28968]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:37.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:37 smithi115 ceph-mon[39520]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:38.971 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.6 2024-09-10T19:23:39.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:38 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:38] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:39.287 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for osd.7 2024-09-10T19:23:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQChnOBmJSgACRAADzpHW/l437TQAq/B6eCxow== 2024-09-10T19:23:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC3muBmLa2CLhAAllfgdqs9OAPRbL0y0K20DA== == AQChnOBmJSgACRAADzpHW/l437TQAq/B6eCxow== ']' 2024-09-10T19:23:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:23:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for osd.7' 2024-09-10T19:23:39.288 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T19:23:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== 2024-09-10T19:23:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== 2024-09-10T19:23:39.592 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key osd.7 2024-09-10T19:23:39.592 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:39 smithi115 ceph-mon[28968]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:39.593 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:39 smithi115 ceph-mon[39520]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:39.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:39 smithi150 ceph-mon[33096]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:39.850 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key osd.7 on host 'smithi150' 2024-09-10T19:23:39.861 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== == AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== ']' 2024-09-10T19:23:39.861 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1299102324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2713111602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.478 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.479 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:40 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:23:40.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1299102324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:40.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2713111602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1299102324' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.6"}]: dispatch 2024-09-10T19:23:40.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2713111602' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]: dispatch 2024-09-10T19:23:40.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:40 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "osd.7", "format": "json"}]': finished 2024-09-10T19:23:41.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:41 smithi150 ceph-mon[33096]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:41 smithi150 ceph-mon[33096]: from='client.15336 v1:172.21.15.115:0/814423623' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:41 smithi150 ceph-mon[33096]: Schedule rotate-key daemon osd.7 2024-09-10T19:23:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:41 smithi150 ceph-mon[33096]: Rotating authentication key for osd.7 2024-09-10T19:23:41.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:41 smithi150 ceph-mon[33096]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T19:23:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[28968]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[28968]: from='client.15336 v1:172.21.15.115:0/814423623' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[28968]: Schedule rotate-key daemon osd.7 2024-09-10T19:23:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[28968]: Rotating authentication key for osd.7 2024-09-10T19:23:41.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[28968]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T19:23:41.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[39520]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:41.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[39520]: from='client.15336 v1:172.21.15.115:0/814423623' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "osd.7", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:23:41.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[39520]: Schedule rotate-key daemon osd.7 2024-09-10T19:23:41.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[39520]: Rotating authentication key for osd.7 2024-09-10T19:23:41.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:41 smithi115 ceph-mon[39520]: Reconfiguring daemon osd.7 on smithi150 2024-09-10T19:23:42.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:42 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:42 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:42 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:42 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:42 smithi150 ceph-mon[33096]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[28968]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:42.663 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:23:42.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:42 smithi115 ceph-mon[39520]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:44.863 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T19:23:45.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:44 smithi150 ceph-mon[33096]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:45.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:45 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:45.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:44 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:44] "GET /metrics HTTP/1.1" 200 219801 "" "Prometheus/2.43.0" 2024-09-10T19:23:45.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:44 smithi115 ceph-mon[28968]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:45.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:44 smithi115 ceph-mon[39520]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:45.174 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== 2024-09-10T19:23:45.174 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== == AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== ']' 2024-09-10T19:23:45.174 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:46.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:45 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:46.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:45 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1066209709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:46.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:45 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:46.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:45 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1066209709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:46.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:45 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:46.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:45 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1066209709' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:47.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:46 smithi150 ceph-mon[33096]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:47.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:46 smithi115 ceph-mon[28968]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:47.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:46 smithi115 ceph-mon[39520]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:49.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:48 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:48] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:48 smithi150 ceph-mon[33096]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:49.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:48 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:48 smithi115 ceph-mon[28968]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:49.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:48 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:49.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:48 smithi115 ceph-mon[39520]: pgmap v222: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:49.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:48 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:23:50.176 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T19:23:50.488 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== 2024-09-10T19:23:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== == AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== ']' 2024-09-10T19:23:50.489 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:51.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:50 smithi115 ceph-mon[39520]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:51.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:50 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2119248426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:50 smithi115 ceph-mon[28968]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:51.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:50 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2119248426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:50 smithi150 ceph-mon[33096]: pgmap v223: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:51.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:50 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2119248426' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:53.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:52 smithi115 ceph-mon[28968]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:53.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:52 smithi115 ceph-mon[39520]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:53.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:52 smithi150 ceph-mon[33096]: pgmap v224: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:55.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:54 smithi115 ceph-mon[39520]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:55.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:54 smithi115 ceph-mon[28968]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:55.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:23:54 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:54] "GET /metrics HTTP/1.1" 200 219801 "" "Prometheus/2.43.0" 2024-09-10T19:23:55.383 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:54 smithi150 ceph-mon[33096]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:55.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:23:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:23:55.490 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T19:23:55.803 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== 2024-09-10T19:23:55.804 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== == AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== ']' 2024-09-10T19:23:55.804 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:23:56.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:55 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:56.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:55 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1003999806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:55 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:56.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:55 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1003999806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:56.284 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:55 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:23:56.284 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:55 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1003999806' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:23:57.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:56 smithi115 ceph-mon[28968]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:57.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:56 smithi115 ceph-mon[39520]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:57.304 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:56 smithi150 ceph-mon[33096]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:23:58.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:23:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:23:58] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:23:59.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:23:58 smithi115 ceph-mon[28968]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:59.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:23:58 smithi115 ceph-mon[39520]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:23:59.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:23:58 smithi150 ceph-mon[33096]: pgmap v227: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:24:00.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:00 smithi150 ceph-mon[33096]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:24:00.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:00 smithi115 ceph-mon[28968]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:24:00.664 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:00 smithi115 ceph-mon[39520]: pgmap v228: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:24:00.805 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key osd.7 2024-09-10T19:24:01.118 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for mgr.y 2024-09-10T19:24:01.119 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC7nOBmPVqXNxAAMo0SKjORWLJAl4Mbaan3+g== 2024-09-10T19:24:01.119 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQDPmuBmESPBFRAApidbUr+IVHomWUZwOxoeEg== == AQC7nOBmPVqXNxAAMo0SKjORWLJAl4Mbaan3+g== ']' 2024-09-10T19:24:01.119 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:24:01.119 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for mgr.y' 2024-09-10T19:24:01.119 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.y 2024-09-10T19:24:01.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:01 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1064260115' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:24:01.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:01 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1064260115' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:24:01.426 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== 2024-09-10T19:24:01.426 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== 2024-09-10T19:24:01.426 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key mgr.y 2024-09-10T19:24:01.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:01 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1064260115' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "osd.7"}]: dispatch 2024-09-10T19:24:01.681 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key mgr.y on host 'smithi115' 2024-09-10T19:24:01.692 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== == AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== ']' 2024-09-10T19:24:01.693 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/3405303056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='client.15384 v1:172.21.15.115:0/1340942210' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: Schedule rotate-key daemon mgr.y 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: Rotating authentication key for mgr.y 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[28968]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:24:02.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/3405303056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='client.15384 v1:172.21.15.115:0/1340942210' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: Schedule rotate-key daemon mgr.y 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: Rotating authentication key for mgr.y 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.416 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:02.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:02 smithi115 ceph-mon[39520]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/3405303056' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='client.15384 v1:172.21.15.115:0/1340942210' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.y", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: Schedule rotate-key daemon mgr.y 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: Rotating authentication key for mgr.y 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:02.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.635 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:02.635 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:02.635 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:02.635 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:02.635 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:02 smithi150 ceph-mon[33096]: pgmap v229: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.334+0000 7f7633be4640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.336+0000 7f7625b48640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-y/keyring', '-n', 'mgr.y', 'tell', 'mgr.y', 'rotate-key', '-i', '-']: 2024-09-10T19:24:03.329+0000 7f02e7fff640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13218660802792515032 != 18374858748799134293 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- 172.21.15.115:0/2963392580 >> v1:172.21.15.115:6790/0 conn(0x7f02f0108210 legacy=0x7f02f010a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- 172.21.15.115:0/2963392580 shutdown_connections 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- 172.21.15.115:0/2963392580 >> 172.21.15.115:0/2963392580 conn(0x7f02f006a6b0 msgr2=0x7f02f006cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- 172.21.15.115:0/2963392580 shutdown_connections 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- 172.21.15.115:0/2963392580 wait complete. 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 Processor -- start 2024-09-10T19:24:03.484 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.330+0000 7f02f74c6640 1 -- start start 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f74c6640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f01159e0 con 0x7f02f010be00 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f74c6640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f01212b0 con 0x7f02f0108210 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f74c6640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f0122450 con 0x7f02f0104710 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f5262640 1 --1- >> v1:172.21.15.115:6790/0 conn(0x7f02f0104710 0x7f02f0114ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.115:6790/0 says I am v1:172.21.15.115:54770/0 (socket says 172.21.15.115:54770) 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f5262640 1 -- 172.21.15.115:0/4057911382 learned_addr learned my addr 172.21.15.115:0/4057911382 (peer_addr_for_me v1:172.21.15.115:0/0) 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423922484 0 0) 0x7f02f01159e0 con 0x7f02f010be00 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02dc0034e0 con 0x7f02f010be00 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143488812 0 0) 0x7f02f0122450 con 0x7f02f0104710 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02f01159e0 con 0x7f02f0104710 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896531504 0 0) 0x7f02f01212b0 con 0x7f02f0108210 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02f0122450 con 0x7f02f0108210 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 2813331965 0 0) 0x7f02dc0034e0 con 0x7f02f010be00 2024-09-10T19:24:03.485 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02f01212b0 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 419317350 0 0) 0x7f02e8001cf0 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.2 v1:172.21.15.115:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 2150187879 0 0) 0x7f02f01159e0 con 0x7f02f0104710 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f02dc0034e0 con 0x7f02f0104710 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.2 v1:172.21.15.115:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 419317350 0 0) 0x7f02e0001800 con 0x7f02f0104710 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 564+0+0 (unknown 3393701555 0 0) 0x7f02f01212b0 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 >> v1:172.21.15.115:6790/0 conn(0x7f02f0104710 legacy=0x7f02f0114ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 >> v1:172.21.15.150:6789/0 conn(0x7f02f0108210 legacy=0x7f02f011b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f01235f0 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f02f01159e0 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f74c6640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f0122640 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02f74c6640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f0122b10 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 5 ==== config(37 keys) v1 ==== 1727+0+0 (unknown 2874922459 0 0) 0x7f02e8002820 con 0x7f02f010be00 2024-09-10T19:24:03.486 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 419317350 0 0) 0x7f02e8003fb0 con 0x7f02f010be00 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 1292601260 0 0) 0x7f02f0122b10 con 0x7f02f010be00 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.331+0000 7f02e5ffb640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13218660802792515032 != 18374858748799134293 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.333+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 8 ==== mgrmap(e 21) v1 ==== 90804+0+0 (unknown 2271351135 0 0) 0x7f02e80020f0 con 0x7f02f010be00 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.333+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 --> v1:172.21.15.115:6800/338981440 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f02dc073b90 con 0x7f02dc06fed0 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.333+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mon.0 v1:172.21.15.115:6789/0 9 ==== osd_map(64..64 src has 1..64) v4 ==== 6284+0+0 (unknown 1709122411 0 0) 0x7f02e8087b30 con 0x7f02f010be00 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.334+0000 7f02e5ffb640 1 -- 172.21.15.115:0/4057911382 <== mgr.34493 v1:172.21.15.115:6800/338981440 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (unknown 3575743073 0 0) 0x7f02dc073b90 con 0x7f02dc06fed0 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: Error EACCES: problem getting command descriptions from mgr.y 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.348+0000 7f764a5f3640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setuser ceph since I am not root 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: ignoring --setgroup ceph since I am not root 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-mgr[29200]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 13218660802792515032 != 18374858748799134293 2024-09-10T19:24:03.487 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.480+0000 7fb48e16dec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:24:03.632 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE Bus STOPPING 2024-09-10T19:24:03.860 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.535+0000 7fb48e16dec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:03.860 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.599+0000 7fb48e16dec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:24:03.860 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.723+0000 7fb48e16dec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:03.935 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:24:03.935 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE Bus STOPPED 2024-09-10T19:24:03.935 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE Bus STARTING 2024-09-10T19:24:03.935 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE Serving on http://:::9283 2024-09-10T19:24:03.935 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:03] ENGINE Bus STARTED 2024-09-10T19:24:04.126 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:03 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:03.861+0000 7fb48e16dec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:24:04.126 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:04.124+0000 7fb48e16dec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:24:04.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:04.187+0000 7fb48e16dec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:24:04.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:04.371+0000 7fb48e16dec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: Activating manager daemon x 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:24:04.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: mgrmap e22: x(active, starting, since 0.00626441s) 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:04.415 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: Manager daemon x is now available 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.416 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: Activating manager daemon x 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: mgrmap e22: x(active, starting, since 0.00626441s) 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:04.417 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:04.418 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: Manager daemon x is now available 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.419 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:04 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.34493 v1:172.21.15.115:0/1110941614' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: Activating manager daemon x 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-10T19:24:04.619 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: mgrmap e22: x(active, starting, since 0.00626441s) 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:04.620 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: Manager daemon x is now available 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.621 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:04.784+0000 7f7d37bb8640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:24:04] ENGINE Error in HTTPServer.serve 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: Traceback (most recent call last): 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: self._connections.run(self.expiration_interval) 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: self._run(expiration_interval) 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:24:04.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: s = self.context.wrap_socket( 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: return self.sslsocket_class._create( 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: self.do_handshake() 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: self._sslobj.do_handshake() 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:24:04.884 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:05.354 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:24:05.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: mgrmap e23: x(active, since 1.00956s) 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: [10/Sep/2024:19:24:04] ENGINE Bus STARTING 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: [10/Sep/2024:19:24:04] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: [10/Sep/2024:19:24:04] ENGINE Bus STARTED 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: [10/Sep/2024:19:24:04] ENGINE Error in HTTPServer.serve 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: Traceback (most recent call last): 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._run(expiration_interval) 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:24:05.633 INFO:journalctl@ceph.mon.b.smithi150.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: s = self.context.wrap_socket( 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: return self.sslsocket_class._create( 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: self.do_handshake() 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: self._sslobj.do_handshake() 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:05.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:05 smithi150 ceph-mon[33096]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:05.282+0000 7fb48e16dec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:05.352+0000 7fb48e16dec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: mgrmap e23: x(active, since 1.00956s) 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: [10/Sep/2024:19:24:04] ENGINE Bus STARTING 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: [10/Sep/2024:19:24:04] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T19:24:05.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: [10/Sep/2024:19:24:04] ENGINE Bus STARTED 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: [10/Sep/2024:19:24:04] ENGINE Error in HTTPServer.serve 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:24:05.641 INFO:journalctl@ceph.mon.c.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout: self.do_handshake() 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[39520]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: mgrmap e23: x(active, since 1.00956s) 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: [10/Sep/2024:19:24:04] ENGINE Bus STARTING 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: [10/Sep/2024:19:24:04] ENGINE Serving on https://172.21.15.150:7150 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: [10/Sep/2024:19:24:04] ENGINE Bus STARTED 2024-09-10T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: [10/Sep/2024:19:24:04] ENGINE Error in HTTPServer.serve 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: Traceback (most recent call last): 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._connections.run(self.expiration_interval) 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._run(expiration_interval) 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: s = self.context.wrap_socket( 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:24:05.643 INFO:journalctl@ceph.mon.a.smithi115.stdout: return self.sslsocket_class._create( 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout: self.do_handshake() 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout: self._sslobj.do_handshake() 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:05.644 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:05 smithi115 ceph-mon[28968]: from='client.24466 v1:172.21.15.150:0/1604624378' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:24:06.360 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.084+0000 7fb48e16dec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:24:06.361 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.197+0000 7fb48e16dec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:24:06.361 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.321+0000 7fb48e16dec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:06.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:06 smithi150 ceph-mon[33096]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:06.638 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-mon[39520]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:06.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-mon[28968]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:06.639 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.385+0000 7fb48e16dec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:24:06.639 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.511+0000 7fb48e16dec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:24:06.639 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.579+0000 7fb48e16dec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:24:06.639 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.637+0000 7fb48e16dec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:24:06.694 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.y 2024-09-10T19:24:06.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.785+0000 7fb48e16dec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:24:06.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.854+0000 7fb48e16dec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:24:07.051 INFO:teuthology.orchestra.run.smithi115.stdout:rotating key for mgr.x 2024-09-10T19:24:07.052 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDRnOBmNRd3LRAAVV1N3GQL3HLg42Hd7PzZTQ== 2024-09-10T19:24:07.052 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQC6meBmdSeFNBAAVM5t+tOT7b414Q6x1Yll5g== == AQDRnOBmNRd3LRAAVV1N3GQL3HLg42Hd7PzZTQ== ']' 2024-09-10T19:24:07.052 INFO:teuthology.orchestra.run.smithi115.stderr:+ for f in osd.0 osd.1 osd.2 osd.3 osd.4 osd.5 osd.6 osd.7 mgr.y mgr.x 2024-09-10T19:24:07.052 INFO:teuthology.orchestra.run.smithi115.stderr:+ echo 'rotating key for mgr.x' 2024-09-10T19:24:07.052 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:24:07.364 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:06 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:06.922+0000 7fb48e16dec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:24:07.364 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:07.008+0000 7fb48e16dec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:07.364 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:07.070+0000 7fb48e16dec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:24:07.364 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:07.132+0000 7fb48e16dec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:24:07.370 INFO:teuthology.orchestra.run.smithi115.stderr:+ K=AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== 2024-09-10T19:24:07.370 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== 2024-09-10T19:24:07.370 INFO:teuthology.orchestra.run.smithi115.stderr:+ ceph orch daemon rotate-key mgr.x 2024-09-10T19:24:07.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: mgrmap e24: x(active, since 3s) 2024-09-10T19:24:07.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1311641480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:07 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/1004046412' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:07.639 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled to rotate-key mgr.x on host 'smithi150' 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: mgrmap e24: x(active, since 3s) 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1311641480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.639 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/1004046412' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: mgrmap e24: x(active, since 3s) 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1311641480' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.y"}]: dispatch 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:07.640 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/1004046412' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:07.649 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== == AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== ']' 2024-09-10T19:24:07.649 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:07.915+0000 7fb48e16dec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:24:07] ENGINE Bus STARTING 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: CherryPy Checker: 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: The Application mounted at '' has an empty config. 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:07 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:24:08] ENGINE Serving on http://:::9283 2024-09-10T19:24:08.366 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[29196]: [10/Sep/2024:19:24:08] ENGINE Bus STARTED 2024-09-10T19:24:08.632 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.633 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='client.35348 v1:172.21.15.115:0/4134705571' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: Schedule rotate-key daemon mgr.x 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: Standby manager daemon y started 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.634 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.663 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='client.35348 v1:172.21.15.115:0/4134705571' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: Schedule rotate-key daemon mgr.x 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.664 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: Standby manager daemon y started 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.665 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi115 to 3248M 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='client.35348 v1:172.21.15.115:0/4134705571' entity='client.admin' cmd=[{"prefix": "orch daemon", "action": "rotate-key", "name": "mgr.x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: Schedule rotate-key daemon mgr.x 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.666 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: Standby manager daemon y started 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.115:0/4133544667' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:08.667 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:08 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:09.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ::ffff:172.21.15.150 - - [10/Sep/2024:19:24:08] "GET /metrics HTTP/1.1" 200 34931 "" "Prometheus/2.43.0" 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.124 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.125 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.126 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.127 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.128 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:09 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: mgrmap e25: x(active, since 5s), standbys: y 2024-09-10T19:24:10.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Adjusting osd_memory_target on smithi150 to 3248M 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:10.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi115:/etc/ceph/ceph.conf 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi150:/etc/ceph/ceph.conf 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi115:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: Updating smithi150:/etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:10.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:09 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:11.130 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: Rotating authentication key for mgr.y 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:11.131 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[39520]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: Rotating authentication key for mgr.y 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.132 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:10 smithi115 ceph-mon[28968]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: Updating smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: Updating smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: Rotating authentication key for mgr.y 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]: dispatch 2024-09-10T19:24:11.133 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.y", "format": "json"}]': finished 2024-09-10T19:24:11.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:11.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:11.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:11.134 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:10 smithi150 ceph-mon[33096]: Reconfiguring daemon mgr.y on smithi115 2024-09-10T19:24:11.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:11 smithi115 systemd[1]: Stopping Ceph mgr.y for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:12 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:12 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:12 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.163 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:12 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:11 smithi115 podman[80515]: 2024-09-10 19:24:11.898786079 +0000 UTC m=+0.119128717 container stop ac1b00b03934eeb528edaeaabab465550438d900a12c790dfa57ff76961149bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T19:24:12.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:11 smithi115 podman[80515]: 2024-09-10 19:24:11.924855778 +0000 UTC m=+0.145198419 container died ac1b00b03934eeb528edaeaabab465550438d900a12c790dfa57ff76961149bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T19:24:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:12 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:12 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:12.607 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 podman[80515]: 2024-09-10 19:24:12.252656084 +0000 UTC m=+0.472998724 container cleanup ac1b00b03934eeb528edaeaabab465550438d900a12c790dfa57ff76961149bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:12.607 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 bash[80515]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:24:12.607 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 podman[80529]: 2024-09-10 19:24:12.412412204 +0000 UTC m=+0.507927778 container remove ac1b00b03934eeb528edaeaabab465550438d900a12c790dfa57ff76961149bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:12.607 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:24:12.651 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 systemd[1]: Stopped Ceph mgr.y for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service: Consumed 42.773s CPU time. 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 systemd[1]: Starting Ceph mgr.y for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 podman[80620]: 2024-09-10 19:24:12.735490071 +0000 UTC m=+0.018129620 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T19:24:12.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:12 smithi115 podman[80620]: 2024-09-10 19:24:12.855057728 +0000 UTC m=+0.137697257 container create 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:12.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== 2024-09-10T19:24:12.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== == AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== ']' 2024-09-10T19:24:12.970 INFO:teuthology.orchestra.run.smithi115.stderr:+ sleep 5 2024-09-10T19:24:13.268 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-mon[28968]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:24:13.268 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/4035815792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:13.268 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-mon[39520]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:24:13.268 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/4035815792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 podman[80620]: 2024-09-10 19:24:13.09106038 +0000 UTC m=+0.373699915 container init 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 podman[80620]: 2024-09-10 19:24:13.094197852 +0000 UTC m=+0.376837382 container start 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-mgr[80660]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 15855031217488436990 != 18374858748799134293 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 bash[80620]: 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 systemd[1]: Started Ceph mgr.y for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.211+0000 7f3faadc4ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:24:13.269 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.266+0000 7f3faadc4ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:13.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:13 smithi150 ceph-mon[33096]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T19:24:13.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:13 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/4035815792' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:13.589 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.331+0000 7f3faadc4ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:24:13.589 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.454+0000 7f3faadc4ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:13.849 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.587+0000 7f3faadc4ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:24:14.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.847+0000 7f3faadc4ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:24:14.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:13 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:13.907+0000 7f3faadc4ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:24:14.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:14.068+0000 7f3faadc4ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: Rotating authentication key for mgr.x 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:14.516 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:14.517 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mon[33096]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T19:24:14.536 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.536 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.536 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: Rotating authentication key for mgr.x 2024-09-10T19:24:14.536 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[28968]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: Rotating authentication key for mgr.x 2024-09-10T19:24:14.537 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create-pending", "entity": "mgr.x", "format": "json"}]': finished 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:24:14.538 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-mon[39520]: Reconfiguring daemon mgr.x on smithi150 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.675+0000 7f7d6d423640 -1 mgr.server reply reply (13) Permission denied access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.677+0000 7f7d6ac1e640 -1 log_channel(cephadm) log [ERR] : Non-zero return from ['ceph', '-k', '/var/lib/ceph/mgr/ceph-x/keyring', '-n', 'mgr.x', 'tell', 'mgr.x', 'rotate-key', '-i', '-']: 2024-09-10T19:24:14.671+0000 7f34f2ffd640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16140166691615557964 != 18374858748799134293 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.671+0000 7f34f9b17640 1 -- 172.21.15.150:0/2176989358 >> v1:172.21.15.150:6789/0 conn(0x7f34f410b540 legacy=0x7f34f410d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.671+0000 7f34f9b17640 1 -- 172.21.15.150:0/2176989358 shutdown_connections 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.671+0000 7f34f9b17640 1 -- 172.21.15.150:0/2176989358 >> 172.21.15.150:0/2176989358 conn(0x7f34f4069e80 msgr2=0x7f34f406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- 172.21.15.150:0/2176989358 shutdown_connections 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- 172.21.15.150:0/2176989358 wait complete. 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 Processor -- start 2024-09-10T19:24:14.825 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- start start 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- --> v1:172.21.15.115:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f4115b40 con 0x7f34f41079e0 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f4121440 con 0x7f34f406e610 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f9b17640 1 -- --> v1:172.21.15.115:6790/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f41225e0 con 0x7f34f410b540 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f37fe640 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f34f406e610 0x7f34f4115050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:53632/0 (socket says 172.21.15.150:53632) 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f37fe640 1 -- 172.21.15.150:0/1574355918 learned_addr learned my addr 172.21.15.150:0/1574355918 (peer_addr_for_me v1:172.21.15.150:0/0) 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059601146 0 0) 0x7f34f4121440 con 0x7f34f406e610 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.672+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d40034e0 con 0x7f34f406e610 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.0 v1:172.21.15.115:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156316860 0 0) 0x7f34f4115b40 con 0x7f34f41079e0 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.115:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f4121440 con 0x7f34f41079e0 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 748+0+0 (unknown 414734047 0 0) 0x7f34d40034e0 con 0x7f34f406e610 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34f4115b40 con 0x7f34f406e610 2024-09-10T19:24:14.826 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 419317350 0 0) 0x7f34e4002940 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.2 v1:172.21.15.115:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2199159481 0 0) 0x7f34f41225e0 con 0x7f34f410b540 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.115:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d40034e0 con 0x7f34f410b540 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 564+0+0 (unknown 160880837 0 0) 0x7f34f4115b40 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 >> v1:172.21.15.115:6790/0 conn(0x7f34f410b540 legacy=0x7f34f411ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 >> v1:172.21.15.115:6789/0 conn(0x7f34f41079e0 legacy=0x7f34f411b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4123780 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- auth(proto 2 2 bytes epoch 0) v1 -- 0x7f34f41225e0 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f9b17640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f4120490 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f9b17640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f4120960 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 5 ==== config(37 keys) v1 ==== 1727+0+0 (unknown 2874922459 0 0) 0x7f34e4001ae0 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 419317350 0 0) 0x7f34e4006650 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 7 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 885255241 0 0) 0x7f34f4120960 con 0x7f34f406e610 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.673+0000 7f34f0ff9640 0 cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16140166691615557964 != 18374858748799134293 2024-09-10T19:24:14.827 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.674+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 8 ==== mgrmap(e 25) v1 ==== 90804+0+0 (unknown 2801992198 0 0) 0x7f34e4005780 con 0x7f34f406e610 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.675+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 --> v1:172.21.15.150:6816/1012146001 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34d4073d40 con 0x7f34d4070080 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.675+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mon.1 v1:172.21.15.150:6789/0 9 ==== osd_map(65..65 src has 1..65) v4 ==== 6499+0+0 (unknown 2087658047 0 0) 0x7f34e408a2a0 con 0x7f34f406e610 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.675+0000 7f34f0ff9640 1 -- 172.21.15.150:0/1574355918 <== mgr.24514 v1:172.21.15.150:6816/1012146001 1 ==== command_reply(tid 0: -13 access denied: does your client key have mgr caps? See http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication) v1 ==== 134+0+0 (unknown 3575743073 0 0) 0x7f34d4073d40 con 0x7f34d4070080 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: Error EACCES: problem getting command descriptions from mgr.x 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.691+0000 7f7d9004a640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ignoring --setuser ceph since I am not root 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: ignoring --setgroup ceph since I am not root 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mgr[34938]: -- 172.21.15.150:0/3099294824 <== mon.1 v1:172.21.15.150:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 4212642619 0 0) 0x55cfb181a820 con 0x55cfb1826c00 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-mgr[34938]: cephx client: could not set rotating key: decode_decrypt failed. error:bad magic in decode_decrypt, 16140166691615557964 != 18374858748799134293 2024-09-10T19:24:14.828 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.822+0000 7f14e1159ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T19:24:15.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.879+0000 7f14e1159ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:15.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:14.944+0000 7f14e1159ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T19:24:15.133 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:15.065+0000 7f14e1159ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:15.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:14 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:14.961+0000 7f3faadc4ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:24:15.413 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:15.029+0000 7f3faadc4ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:24:15.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[28968]: mgrmap e26: y(active, starting, since 0.011069s) 2024-09-10T19:24:15.414 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: Activating manager daemon y 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:24:15.415 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-mon[39520]: mgrmap e26: y(active, starting, since 0.011069s) 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:15.202+0000 7f14e1159ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: from='mgr.24514 v1:172.21.15.150:0/95411128' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-10T19:24:15.461 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: Activating manager daemon y 2024-09-10T19:24:15.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T19:24:15.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: from='mgr.24514 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-10T19:24:15.462 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-mon[33096]: mgrmap e26: y(active, starting, since 0.011069s) 2024-09-10T19:24:15.868 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:15.755+0000 7f3faadc4ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:24:15.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:15.459+0000 7f14e1159ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T19:24:15.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:15.520+0000 7f14e1159ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T19:24:15.883 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:15.679+0000 7f14e1159ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T19:24:16.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:15.867+0000 7f3faadc4ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:24:16.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:15 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:15.987+0000 7f3faadc4ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:16.164 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.045+0000 7f3faadc4ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:24:16.430 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.171+0000 7f3faadc4ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:24:16.430 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.238+0000 7f3faadc4ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:24:16.430 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.294+0000 7f3faadc4ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:24:16.430 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.428+0000 7f3faadc4ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:24:16.694 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.492+0000 7f3faadc4ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:24:16.695 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.556+0000 7f3faadc4ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:24:16.695 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.636+0000 7f3faadc4ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:16.695 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.693+0000 7f3faadc4ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:24:16.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:16.517+0000 7f14e1159ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T19:24:16.882 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:16.581+0000 7f14e1159ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T19:24:17.163 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:16 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:16.750+0000 7f3faadc4ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:24:17.581 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.287+0000 7f14e1159ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T19:24:17.581 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.399+0000 7f14e1159ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T19:24:17.581 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.521+0000 7f14e1159ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T19:24:17.581 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.579+0000 7f14e1159ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T19:24:17.829 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:17.499+0000 7f3faadc4ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STARTING 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: CherryPy Checker: 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: The Application mounted at '' has an empty config. 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Serving on http://:::9283 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STARTED 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STOPPING 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STOPPED 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STARTING 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Serving on http://:::9283 2024-09-10T19:24:17.830 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:17] ENGINE Bus STARTED 2024-09-10T19:24:17.837 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.709+0000 7f14e1159ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T19:24:17.838 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.777+0000 7f14e1159ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T19:24:17.838 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.836+0000 7f14e1159ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T19:24:17.973 INFO:teuthology.orchestra.run.smithi115.stderr:++ ceph auth get-key mgr.x 2024-09-10T19:24:18.115 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:17.977+0000 7f14e1159ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T19:24:18.115 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:18.044+0000 7f14e1159ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T19:24:18.115 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:18.113+0000 7f14e1159ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: Active manager daemon y restarted 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: Activating manager daemon y 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.? v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: mgrmap e27: y(active, starting, since 0.0063055s) 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:18.116 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:18.117 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: Manager daemon y is now available 2024-09-10T19:24:18.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:18.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:18.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:18.118 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:17 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: Active manager daemon y restarted 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: Activating manager daemon y 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.? v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: mgrmap e27: y(active, starting, since 0.0063055s) 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:18.165 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: Manager daemon y is now available 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:18.166 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: Active manager daemon y restarted 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: Activating manager daemon y 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.? v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: mgrmap e27: y(active, starting, since 0.0063055s) 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T19:24:18.167 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: Manager daemon y is now available 2024-09-10T19:24:18.168 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:24:18.169 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:24:18.169 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T19:24:18.169 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:17 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T19:24:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:+ NK=AQDdnOBm7k43DBAALyzc+jEKKfYQkzp5Oz1jSw== 2024-09-10T19:24:18.310 INFO:teuthology.orchestra.run.smithi115.stderr:+ '[' AQAnmuBmKaMiAxAAzs67bXtPtnsBg6pfosxsoA== == AQDdnOBm7k43DBAALyzc+jEKKfYQkzp5Oz1jSw== ']' 2024-09-10T19:24:18.382 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:18.200+0000 7f14e1159ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T19:24:18.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:18.261+0000 7f14e1159ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T19:24:18.383 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:18.326+0000 7f14e1159ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T19:24:18.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug there is no tcmu-runner data available 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:18.828+0000 7f3f4d945640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:19:24:18] ENGINE Error in HTTPServer.serve 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: Traceback (most recent call last): 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: self._connections.run(self.expiration_interval) 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: self._run(expiration_interval) 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T19:24:18.899 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: s = self.context.wrap_socket( 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: return self.sslsocket_class._create( 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: self.do_handshake() 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: self._sslobj.do_handshake() 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T19:24:18.900 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:18.951 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T19:24:18.961 INFO:tasks.cephadm:Teardown begin 2024-09-10T19:24:18.961 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:18.990 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:24:19.018 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T19:24:19.018 DEBUG:teuthology.orchestra.run.smithi115:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:24:19.045 DEBUG:teuthology.orchestra.run.smithi150:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:24:19.073 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T19:24:19.073 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T19:24:19.074 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a 2024-09-10T19:24:19.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-mon[33096]: from='client.? v1:172.21.15.115:0/2639770254' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:19.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-mon[33096]: mgrmap e28: y(active, since 1.00802s) 2024-09-10T19:24:19.132 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:18 smithi150 ceph-mon[33096]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' 2024-09-10T19:24:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[28968]: from='client.? v1:172.21.15.115:0/2639770254' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:19.163 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[28968]: mgrmap e28: y(active, since 1.00802s) 2024-09-10T19:24:19.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[28968]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' 2024-09-10T19:24:19.164 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:19 smithi115 systemd[1]: Stopping Ceph mon.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:19.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[39520]: from='client.? v1:172.21.15.115:0/2639770254' entity='client.admin' cmd=[{"prefix": "auth get-key", "entity": "mgr.x"}]: dispatch 2024-09-10T19:24:19.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[39520]: mgrmap e28: y(active, since 1.00802s) 2024-09-10T19:24:19.164 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:18 smithi115 ceph-mon[39520]: from='mgr.35360 v1:172.21.15.115:0/4017514433' entity='mgr.y' 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:19.131+0000 7f14e1159ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:19] ENGINE Bus STARTING 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: CherryPy Checker: 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: The Application mounted at '' has an empty config. 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:19] ENGINE Serving on http://:::9283 2024-09-10T19:24:19.625 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x[34934]: [10/Sep/2024:19:24:19] ENGINE Bus STARTED 2024-09-10T19:24:20.120 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a[28964]: 2024-09-10T19:24:19.801+0000 7f459cdb3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a[28964]: 2024-09-10T19:24:19.801+0000 7f459cdb3640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:19 smithi115 podman[81189]: 2024-09-10 19:24:19.8638812 +0000 UTC m=+0.174498867 container died 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE Bus STOPPING 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE Bus STOPPED 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE Bus STARTING 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE Serving on http://:::9283 2024-09-10T19:24:20.121 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:19 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: [10/Sep/2024:19:24:19] ENGINE Bus STARTED 2024-09-10T19:24:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:20 smithi115 podman[81189]: 2024-09-10 19:24:20.21875417 +0000 UTC m=+0.529371841 container cleanup 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T19:24:20.413 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:20 smithi115 bash[81189]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a 2024-09-10T19:24:20.857 INFO:journalctl@ceph.mon.a.smithi115.stdout:Sep 10 19:24:20 smithi115 podman[81263]: 2024-09-10 19:24:20.495019398 +0000 UTC m=+0.626390911 container remove 4f5ce18c6ec5139af6266db83d6e421da19c1b433ba9cbae9b10933d042909b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T19:24:21.005 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T19:24:21.072 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:21.073 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T19:24:21.073 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T19:24:21.073 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.c 2024-09-10T19:24:21.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:21 smithi115 systemd[1]: Stopping Ceph mon.c for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:22.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c[39516]: 2024-09-10T19:24:22.112+0000 7f3a49df9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:22.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:22 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c[39516]: 2024-09-10T19:24:22.112+0000 7f3a49df9640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:24:22.413 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:22 smithi115 podman[81579]: 2024-09-10 19:24:22.154748356 +0000 UTC m=+0.118839814 container died 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:22.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:22 smithi115 podman[81579]: 2024-09-10 19:24:22.504450139 +0000 UTC m=+0.468541596 container cleanup 05eb75da8ca8161ceb8ae2539286a5654adf7325b2f77871e86cf2ebc6f94432 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T19:24:22.834 INFO:journalctl@ceph.mon.c.smithi115.stdout:Sep 10 19:24:22 smithi115 bash[81579]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-c 2024-09-10T19:24:22.993 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T19:24:23.031 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:23.031 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T19:24:23.031 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T19:24:23.032 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.b 2024-09-10T19:24:23.382 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 systemd[1]: Stopping Ceph mon.b for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:23.789 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-b[33086]: 2024-09-10T19:24:23.428+0000 7f05f6cf5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:23.789 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-b[33086]: 2024-09-10T19:24:23.428+0000 7f05f6cf5640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:24:23.789 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 podman[69911]: 2024-09-10 19:24:23.501077457 +0000 UTC m=+0.276481718 container died ddf1064b8accb1bd25e6deadad99494e0fc502e4e1d5c3d0b7076c3c3dba4c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T19:24:24.123 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 podman[69911]: 2024-09-10 19:24:23.9043451 +0000 UTC m=+0.679749361 container cleanup ddf1064b8accb1bd25e6deadad99494e0fc502e4e1d5c3d0b7076c3c3dba4c04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T19:24:24.123 INFO:journalctl@ceph.mon.b.smithi150.stdout:Sep 10 19:24:23 smithi150 bash[69911]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mon-b 2024-09-10T19:24:24.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T19:24:24.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:24.312 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T19:24:24.312 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T19:24:24.313 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y 2024-09-10T19:24:24.629 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 systemd[1]: Stopping Ceph mgr.y for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:24.913 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:24.626+0000 7f3fa6d59640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:24.914 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y[80654]: 2024-09-10T19:24:24.626+0000 7f3fa6d59640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T19:24:24.914 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 podman[81699]: 2024-09-10 19:24:24.658484981 +0000 UTC m=+0.153483889 container died 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:25.407 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 podman[81699]: 2024-09-10 19:24:24.994487014 +0000 UTC m=+0.489485919 container cleanup 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:24:25.407 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:24 smithi115 bash[81699]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:24:25.407 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Sep 10 19:24:25 smithi115 podman[81713]: 2024-09-10 19:24:25.170704369 +0000 UTC m=+0.512586297 container remove 43f356fc0de381498002a5226dabe01d80b38545e7e71dd79f8e17cd5eb4ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T19:24:25.428 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T19:24:25.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:25.463 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T19:24:25.463 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T19:24:25.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x 2024-09-10T19:24:25.805 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:25 smithi150 systemd[1]: Stopping Ceph mgr.x for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:26.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:25 smithi150 podman[70032]: 2024-09-10 19:24:25.80464187 +0000 UTC m=+0.141842893 container stop 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T19:24:26.132 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:25 smithi150 podman[70032]: 2024-09-10 19:24:25.830316979 +0000 UTC m=+0.167517992 container died 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:24:26.436 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 podman[70032]: 2024-09-10 19:24:26.259113969 +0000 UTC m=+0.596314982 container cleanup 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:26.436 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 bash[70032]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x 2024-09-10T19:24:26.687 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 podman[70045]: 2024-09-10 19:24:26.435401998 +0000 UTC m=+0.620921101 container remove 377a059a7e3efa5a0ccbcfa23121abb9a863d7696a2916904b589d030ce83d29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:26.687 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:24:26.688 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-10T19:24:26.688 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 systemd[1]: Stopped Ceph mgr.x for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:24:26.688 INFO:journalctl@ceph.mgr.x.smithi150.stdout:Sep 10 19:24:26 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x.service: Consumed 17.327s CPU time. 2024-09-10T19:24:26.690 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T19:24:26.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:26.726 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T19:24:26.726 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T19:24:26.726 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.0 2024-09-10T19:24:27.050 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:26 smithi115 systemd[1]: Stopping Ceph osd.0 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:27.413 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:27 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0[43377]: 2024-09-10T19:24:27.047+0000 7f86700f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:27.413 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:27 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0[43377]: 2024-09-10T19:24:27.047+0000 7f86700f6640 -1 osd.0 67 *** Got signal Terminated *** 2024-09-10T19:24:27.413 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:27 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0[43377]: 2024-09-10T19:24:27.047+0000 7f86700f6640 -1 osd.0 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:24:32.758 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:32 smithi115 podman[81819]: 2024-09-10 19:24:32.474036946 +0000 UTC m=+5.543652550 container died c9aac436f557d370e5d2d2ed4d3f373d0a33ec9191ccfc83e9c4e04defdfb600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:24:33.100 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:32 smithi115 podman[81819]: 2024-09-10 19:24:32.856384868 +0000 UTC m=+5.926000471 container cleanup c9aac436f557d370e5d2d2ed4d3f373d0a33ec9191ccfc83e9c4e04defdfb600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T19:24:33.100 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:32 smithi115 bash[81819]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0 2024-09-10T19:24:33.405 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81834]: 2024-09-10 19:24:33.099423821 +0000 UTC m=+0.622691356 container remove c9aac436f557d370e5d2d2ed4d3f373d0a33ec9191ccfc83e9c4e04defdfb600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T19:24:33.663 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.307634285 +0000 UTC m=+0.015986590 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:24:33.663 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.408519458 +0000 UTC m=+0.116871807 container create ee401f73c7e8cae4566478af1ad795d5eb9699ddb153920753d52baca941b5e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:24:33.663 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.619176479 +0000 UTC m=+0.327528782 container init ee401f73c7e8cae4566478af1ad795d5eb9699ddb153920753d52baca941b5e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T19:24:33.663 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.622816515 +0000 UTC m=+0.331168828 container start ee401f73c7e8cae4566478af1ad795d5eb9699ddb153920753d52baca941b5e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T19:24:34.140 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.702477199 +0000 UTC m=+0.410829513 container attach ee401f73c7e8cae4566478af1ad795d5eb9699ddb153920753d52baca941b5e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:24:34.140 INFO:journalctl@ceph.osd.0.smithi115.stdout:Sep 10 19:24:33 smithi115 podman[81922]: 2024-09-10 19:24:33.796693627 +0000 UTC m=+0.505045939 container died ee401f73c7e8cae4566478af1ad795d5eb9699ddb153920753d52baca941b5e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T19:24:34.459 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T19:24:34.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:34.499 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T19:24:34.499 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T19:24:34.500 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.1 2024-09-10T19:24:34.690 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:34 smithi115 systemd[1]: Stopping Ceph osd.1 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:35.163 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1[48960]: 2024-09-10T19:24:34.824+0000 7f58c3ab5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:35.163 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1[48960]: 2024-09-10T19:24:34.824+0000 7f58c3ab5640 -1 osd.1 67 *** Got signal Terminated *** 2024-09-10T19:24:35.163 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:34 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1[48960]: 2024-09-10T19:24:34.824+0000 7f58c3ab5640 -1 osd.1 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:24:40.585 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:40 smithi115 podman[82029]: 2024-09-10 19:24:40.280799365 +0000 UTC m=+5.558311736 container died c79353370ee08b1edcd8a3c3782e2b548dbadc6e5ca9863937d2ebc896cd01b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T19:24:40.913 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:40 smithi115 podman[82029]: 2024-09-10 19:24:40.583220964 +0000 UTC m=+5.860733321 container cleanup c79353370ee08b1edcd8a3c3782e2b548dbadc6e5ca9863937d2ebc896cd01b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T19:24:40.913 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:40 smithi115 bash[82029]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1 2024-09-10T19:24:40.913 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:40 smithi115 podman[82044]: 2024-09-10 19:24:40.784566753 +0000 UTC m=+0.502039935 container remove c79353370ee08b1edcd8a3c3782e2b548dbadc6e5ca9863937d2ebc896cd01b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T19:24:41.464 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.100012414 +0000 UTC m=+0.015756212 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:24:41.464 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.201920491 +0000 UTC m=+0.117664280 container create ee42b0e46d17aa37fff6a3ccdff85f2a42682b3a49cc8ee6237e9e0392aa788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T19:24:41.464 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.37950054 +0000 UTC m=+0.295244329 container init ee42b0e46d17aa37fff6a3ccdff85f2a42682b3a49cc8ee6237e9e0392aa788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:41.464 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.382843649 +0000 UTC m=+0.298587444 container start ee42b0e46d17aa37fff6a3ccdff85f2a42682b3a49cc8ee6237e9e0392aa788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T19:24:41.464 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.462701192 +0000 UTC m=+0.378444998 container attach ee42b0e46d17aa37fff6a3ccdff85f2a42682b3a49cc8ee6237e9e0392aa788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:24:41.913 INFO:journalctl@ceph.osd.1.smithi115.stdout:Sep 10 19:24:41 smithi115 podman[82133]: 2024-09-10 19:24:41.559340625 +0000 UTC m=+0.475084415 container died ee42b0e46d17aa37fff6a3ccdff85f2a42682b3a49cc8ee6237e9e0392aa788c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T19:24:42.129 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T19:24:42.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:42.171 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T19:24:42.171 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T19:24:42.171 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.2 2024-09-10T19:24:42.512 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:42 smithi115 systemd[1]: Stopping Ceph osd.2 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:42.913 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2[54777]: 2024-09-10T19:24:42.509+0000 7f6d31220640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:42.913 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2[54777]: 2024-09-10T19:24:42.509+0000 7f6d31220640 -1 osd.2 67 *** Got signal Terminated *** 2024-09-10T19:24:42.913 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:42 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2[54777]: 2024-09-10T19:24:42.509+0000 7f6d31220640 -1 osd.2 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:24:48.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:47 smithi115 podman[82236]: 2024-09-10 19:24:47.909539629 +0000 UTC m=+5.511724354 container died eea1fe6890b0e747725f2c92d689809efe89823e452e6915152644eac1b9e31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:24:48.446 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:48 smithi115 podman[82236]: 2024-09-10 19:24:48.22628726 +0000 UTC m=+5.828471984 container cleanup eea1fe6890b0e747725f2c92d689809efe89823e452e6915152644eac1b9e31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T19:24:48.446 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:48 smithi115 bash[82236]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2 2024-09-10T19:24:48.840 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:48 smithi115 podman[82252]: 2024-09-10 19:24:48.444538024 +0000 UTC m=+0.525783085 container remove eea1fe6890b0e747725f2c92d689809efe89823e452e6915152644eac1b9e31e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:24:49.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:48 smithi115 podman[82341]: 2024-09-10 19:24:48.741867886 +0000 UTC m=+0.015442670 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:24:49.163 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:48 smithi115 podman[82341]: 2024-09-10 19:24:48.861901427 +0000 UTC m=+0.135476200 container create 20ea8fb0c9fa187c37b6ab0150e0cbe9d050ca4233b9653839d0e0e272185dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:49.164 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:49 smithi115 podman[82341]: 2024-09-10 19:24:49.072571535 +0000 UTC m=+0.346146306 container init 20ea8fb0c9fa187c37b6ab0150e0cbe9d050ca4233b9653839d0e0e272185dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T19:24:49.164 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:49 smithi115 podman[82341]: 2024-09-10 19:24:49.075777513 +0000 UTC m=+0.349352284 container start 20ea8fb0c9fa187c37b6ab0150e0cbe9d050ca4233b9653839d0e0e272185dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:24:49.576 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:49 smithi115 podman[82341]: 2024-09-10 19:24:49.183487921 +0000 UTC m=+0.457062701 container attach 20ea8fb0c9fa187c37b6ab0150e0cbe9d050ca4233b9653839d0e0e272185dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:24:49.577 INFO:journalctl@ceph.osd.2.smithi115.stdout:Sep 10 19:24:49 smithi115 podman[82341]: 2024-09-10 19:24:49.251826176 +0000 UTC m=+0.525400957 container died 20ea8fb0c9fa187c37b6ab0150e0cbe9d050ca4233b9653839d0e0e272185dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:24:49.855 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T19:24:49.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:49.895 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T19:24:49.895 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T19:24:49.895 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.3 2024-09-10T19:24:50.163 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:49 smithi115 systemd[1]: Stopping Ceph osd.3 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:50.663 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:50 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3[60613]: 2024-09-10T19:24:50.203+0000 7fa797955640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:50.663 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:50 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3[60613]: 2024-09-10T19:24:50.203+0000 7fa797955640 -1 osd.3 67 *** Got signal Terminated *** 2024-09-10T19:24:50.663 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:50 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3[60613]: 2024-09-10T19:24:50.203+0000 7fa797955640 -1 osd.3 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:24:53.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:53 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:53.360+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:54.611 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:54.316+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:54.882 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:54.610+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:24:55.284 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:54 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:54.986+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:24:55.592 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:55.282+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:55.882 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:55.590+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:24:55.905 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:55 smithi115 podman[82444]: 2024-09-10 19:24:55.636384938 +0000 UTC m=+5.519184343 container died 171a1f48b10c47cac5316b0327d7a2449cf8d5a6e67454b846a3cb4640b97f1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T19:24:56.163 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82444]: 2024-09-10 19:24:56.002980247 +0000 UTC m=+5.885779653 container cleanup 171a1f48b10c47cac5316b0327d7a2449cf8d5a6e67454b846a3cb4640b97f1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T19:24:56.163 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 bash[82444]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3 2024-09-10T19:24:56.328 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:55 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:55.993+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:24:56.569 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82474]: 2024-09-10 19:24:56.254010615 +0000 UTC m=+0.609013736 container remove 171a1f48b10c47cac5316b0327d7a2449cf8d5a6e67454b846a3cb4640b97f1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T19:24:56.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:56 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:56.327+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:56.866 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.471788375 +0000 UTC m=+0.015894054 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:24:56.866 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.579875844 +0000 UTC m=+0.123981565 container create cf4c674ab69ce1f6da7885e80a9732dc8334301398cefe6a2787a2a1bbd3f1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T19:24:56.866 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.790562271 +0000 UTC m=+0.334667949 container init cf4c674ab69ce1f6da7885e80a9732dc8334301398cefe6a2787a2a1bbd3f1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T19:24:56.866 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.793705254 +0000 UTC m=+0.337810936 container start cf4c674ab69ce1f6da7885e80a9732dc8334301398cefe6a2787a2a1bbd3f1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T19:24:56.866 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.865276362 +0000 UTC m=+0.409382058 container attach cf4c674ab69ce1f6da7885e80a9732dc8334301398cefe6a2787a2a1bbd3f1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:24:56.949 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:56 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:56.639+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:24:57.163 INFO:journalctl@ceph.osd.3.smithi115.stdout:Sep 10 19:24:56 smithi115 podman[82564]: 2024-09-10 19:24:56.968662698 +0000 UTC m=+0.512768379 container died cf4c674ab69ce1f6da7885e80a9732dc8334301398cefe6a2787a2a1bbd3f1ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T19:24:57.327 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:56 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:56.947+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:24:57.581 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T19:24:57.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:24:57.620 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T19:24:57.620 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T19:24:57.620 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.4 2024-09-10T19:24:57.632 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:57.325+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:24:57.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:57.354+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:57.925 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:57 smithi150 systemd[1]: Stopping Ceph osd.4 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:24:57.926 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:57.637+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:24:58.321 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:57.923+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:24:58.322 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:57.928+0000 7f4f92cff640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:24:58.322 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:57.928+0000 7f4f92cff640 -1 osd.4 67 *** Got signal Terminated *** 2024-09-10T19:24:58.322 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:57 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:57.928+0000 7f4f92cff640 -1 osd.4 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:24:58.322 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:58.320+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:24:58.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:58.356+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:58.934 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:58.672+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:24:59.300 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:58 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:58.933+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:24:59.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:24:59 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:24:59.365+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:24:59.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:24:59 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:24:59.299+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:25:00.132 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:24:59 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:24:59.945+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:00.133 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:24:59 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:24:59.707+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:00.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:00 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:00.375+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:00.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:00 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:25:00.283+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:25:01.132 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:00 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:00.916+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:01.133 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:00 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:00.691+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:01.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:01.372+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:01.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:25:01.275+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:25:01.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:25:01.275+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.730078+0000 front 2024-09-10T19:24:36.730047+0000 (oldest deadline 2024-09-10T19:25:00.830010+0000) 2024-09-10T19:25:01.954 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:01.696+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:01.954 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:01.696+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:02.230 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:01.952+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:02.230 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:01 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:01.952+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:02.633 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:02.383+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:02.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:25:02.228+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:32.029604+0000 front 2024-09-10T19:24:32.029614+0000 (oldest deadline 2024-09-10T19:24:56.729478+0000) 2024-09-10T19:25:02.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4[37885]: 2024-09-10T19:25:02.228+0000 7f4f8e315640 -1 osd.4 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.730078+0000 front 2024-09-10T19:24:36.730047+0000 (oldest deadline 2024-09-10T19:25:00.830010+0000) 2024-09-10T19:25:02.969 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:02.655+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:02.970 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:02.655+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:03.365 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:02.968+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:03.366 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:02 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:02.968+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:03.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:03.363+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:03.633 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:03 smithi150 podman[70169]: 2024-09-10 19:25:03.416497292 +0000 UTC m=+5.594442596 container died 21312fec2325b7923a6809f6882fc2b67a88ed16b14aa8a9022348f9eb220156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T19:25:04.007 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:03 smithi150 podman[70169]: 2024-09-10 19:25:03.795303211 +0000 UTC m=+5.973248505 container cleanup 21312fec2325b7923a6809f6882fc2b67a88ed16b14aa8a9022348f9eb220156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T19:25:04.007 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:03 smithi150 bash[70169]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4 2024-09-10T19:25:04.008 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:03.664+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:04.008 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:03 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:03.664+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:04.369 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70185]: 2024-09-10 19:25:04.054985047 +0000 UTC m=+0.634439575 container remove 21312fec2325b7923a6809f6882fc2b67a88ed16b14aa8a9022348f9eb220156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T19:25:04.369 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:04.005+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:04.369 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:04.005+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:04.626 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:04.367+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:04.626 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:04.367+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:04.626 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.405643958 +0000 UTC m=+0.099675356 container create c16868076884b3f884ca2ef1e8cf32e5b88adab5c6b083d5ab42910e24181da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T19:25:04.626 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.322440587 +0000 UTC m=+0.016471992 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:25:04.626 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.624957879 +0000 UTC m=+0.318989279 container init c16868076884b3f884ca2ef1e8cf32e5b88adab5c6b083d5ab42910e24181da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T19:25:04.882 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:04.665+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:04.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:04 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:04.665+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:04.883 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.628752143 +0000 UTC m=+0.322783542 container start c16868076884b3f884ca2ef1e8cf32e5b88adab5c6b083d5ab42910e24181da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:25:04.883 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.699972795 +0000 UTC m=+0.394004202 container attach c16868076884b3f884ca2ef1e8cf32e5b88adab5c6b083d5ab42910e24181da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:25:04.883 INFO:journalctl@ceph.osd.4.smithi150.stdout:Sep 10 19:25:04 smithi150 podman[70275]: 2024-09-10 19:25:04.813060958 +0000 UTC m=+0.507092355 container died c16868076884b3f884ca2ef1e8cf32e5b88adab5c6b083d5ab42910e24181da4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T19:25:05.327 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:05.008+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:05.327 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:05.008+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:05.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T19:25:05.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:05.576 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T19:25:05.577 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T19:25:05.577 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.5 2024-09-10T19:25:05.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:05.326+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:05.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:05.326+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:05.917 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:05.700+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:05.918 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:05.700+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:05.918 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 systemd[1]: Stopping Ceph osd.5 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:06.298 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:05.915+0000 7fef78680640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:25:06.298 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:05.915+0000 7fef78680640 -1 osd.5 67 *** Got signal Terminated *** 2024-09-10T19:25:06.298 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:05 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:05.915+0000 7fef78680640 -1 osd.5 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:25:06.298 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:06.022+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:06.298 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:06.022+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:06.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:06.295+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:06.632 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:06.295+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:07.049 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:06.748+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:07.049 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:06 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:06.748+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:07.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:07.283+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:07.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:07.283+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:07.383 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:07.048+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:07.383 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:07.048+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:08.036 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:07.750+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:08.036 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:07 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:07.750+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:08.384 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:08.035+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:08.384 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:08.035+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:08.384 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:08.242+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:08.384 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:08.242+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:09.016 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:08.716+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:09.016 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:08.716+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:09.016 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:08 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:08.716+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:09.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:09.196+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:09.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:09.196+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:09.383 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:09.014+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:09.383 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:09.014+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:10.038 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:09.675+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:10.039 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:09.675+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:10.039 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:09 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:09.675+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:10.382 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:10.037+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.970671+0000 front 2024-09-10T19:24:31.970754+0000 (oldest deadline 2024-09-10T19:24:54.270663+0000) 2024-09-10T19:25:10.382 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5[43430]: 2024-09-10T19:25:10.037+0000 7fef75c9a640 -1 osd.5 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:39.471993+0000 front 2024-09-10T19:24:39.471914+0000 (oldest deadline 2024-09-10T19:25:01.171976+0000) 2024-09-10T19:25:10.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:10.201+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:10.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:10.201+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:11.132 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:10.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:11.132 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:10.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:11.133 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:10 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:10.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:11.617 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:11 smithi150 podman[70378]: 2024-09-10 19:25:11.336910089 +0000 UTC m=+5.521689926 container died 6aba8fe1e9a2b51f392fe54a50e12abd7127d6aa8e4df2b1d270e40d2be2e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:25:11.617 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:11 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:11.165+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:11.617 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:11 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:11.165+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:11.882 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:11 smithi150 podman[70378]: 2024-09-10 19:25:11.714706722 +0000 UTC m=+5.899486552 container cleanup 6aba8fe1e9a2b51f392fe54a50e12abd7127d6aa8e4df2b1d270e40d2be2e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2) 2024-09-10T19:25:11.882 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:11 smithi150 bash[70378]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5 2024-09-10T19:25:11.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:11 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:11.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:11.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:11 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:11.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:11.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:11 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:11.701+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:12.303 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:11 smithi150 podman[70403]: 2024-09-10 19:25:11.991374111 +0000 UTC m=+0.651121850 container remove 6aba8fe1e9a2b51f392fe54a50e12abd7127d6aa8e4df2b1d270e40d2be2e046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T19:25:12.303 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:12 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:12.188+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:12.303 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:12 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:12.188+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:12.595 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70494]: 2024-09-10 19:25:12.205051208 +0000 UTC m=+0.016619548 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:25:12.595 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70494]: 2024-09-10 19:25:12.308925146 +0000 UTC m=+0.120493525 container create 34b1ff320a1092367dbcdf52c07b6254b17e1651a37ace9139b0ec73aee7e733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T19:25:12.596 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70494]: 2024-09-10 19:25:12.519501948 +0000 UTC m=+0.331070285 container init 34b1ff320a1092367dbcdf52c07b6254b17e1651a37ace9139b0ec73aee7e733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:25:12.596 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70494]: 2024-09-10 19:25:12.523031754 +0000 UTC m=+0.334600079 container start 34b1ff320a1092367dbcdf52c07b6254b17e1651a37ace9139b0ec73aee7e733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T19:25:12.596 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70494]: 2024-09-10 19:25:12.594336779 +0000 UTC m=+0.405905114 container attach 34b1ff320a1092367dbcdf52c07b6254b17e1651a37ace9139b0ec73aee7e733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T19:25:12.882 INFO:journalctl@ceph.osd.5.smithi150.stdout:Sep 10 19:25:12 smithi150 podman[70517]: 2024-09-10 19:25:12.72750723 +0000 UTC m=+0.016733667 container died 34b1ff320a1092367dbcdf52c07b6254b17e1651a37ace9139b0ec73aee7e733 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:25:12.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:12 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:12.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:12.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:12 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:12.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:12.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:12 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:12.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:13.307 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:13.139+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:13.307 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:13.139+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:13.307 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:13.139+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:13.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T19:25:13.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:13.407 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T19:25:13.407 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T19:25:13.408 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.6 2024-09-10T19:25:13.632 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 systemd[1]: Stopping Ceph osd.6 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:14.103 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.692+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:14.104 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.692+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:14.104 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.692+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:14.104 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.733+0000 7f6c623de640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:25:14.104 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.733+0000 7f6c623de640 -1 osd.6 67 *** Got signal Terminated *** 2024-09-10T19:25:14.104 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:13 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:13.733+0000 7f6c623de640 -1 osd.6 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:25:14.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:14.101+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:14.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:14.101+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:14.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:14.101+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:15.099 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:14.671+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:15.100 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:14.671+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:15.100 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:14 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:14.671+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:15.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:15.097+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:15.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:15.097+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:15.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:15.097+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:16.084 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:15.705+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:16.085 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:15.705+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:16.085 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:15 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:15.705+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:16.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:16.081+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:16.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:16.081+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:16.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:16.081+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:17.092 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:16.719+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:17.092 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:16.719+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:17.092 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:16.719+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:17.092 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:16 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:16.719+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:50.305942+0000 front 2024-09-10T19:24:50.305926+0000 (oldest deadline 2024-09-10T19:25:16.205972+0000) 2024-09-10T19:25:17.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:17.089+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:17.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:17.089+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:17.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:17.089+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:17.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:17.089+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:18.046 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:17.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:31.103349+0000 front 2024-09-10T19:24:31.103465+0000 (oldest deadline 2024-09-10T19:24:54.003290+0000) 2024-09-10T19:25:18.047 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:17.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:36.904286+0000 front 2024-09-10T19:24:36.904408+0000 (oldest deadline 2024-09-10T19:25:01.604193+0000) 2024-09-10T19:25:18.047 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:17.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:46.905020+0000 front 2024-09-10T19:24:46.905037+0000 (oldest deadline 2024-09-10T19:25:08.004971+0000) 2024-09-10T19:25:18.047 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:17 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6[49340]: 2024-09-10T19:25:17.725+0000 7f6c5f9f8640 -1 osd.6 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:50.305942+0000 front 2024-09-10T19:24:50.305926+0000 (oldest deadline 2024-09-10T19:25:16.205972+0000) 2024-09-10T19:25:18.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:18.044+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:18.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:18.044+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:18.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:18.044+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:18.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:18 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:18.044+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:19.382 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:19 smithi150 podman[70596]: 2024-09-10 19:25:19.208043255 +0000 UTC m=+5.580876459 container died ac8a3d05b787a061ead30d4372436e93cd3989c736a5c6edea8a55174324f992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:25:19.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:19.061+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:19.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:19.061+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:19.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:19.061+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:19.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:19 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:19.061+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:19.744 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:19 smithi150 podman[70596]: 2024-09-10 19:25:19.550748461 +0000 UTC m=+5.923581661 container cleanup ac8a3d05b787a061ead30d4372436e93cd3989c736a5c6edea8a55174324f992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:25:19.744 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:19 smithi150 bash[70596]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6 2024-09-10T19:25:20.036 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:19 smithi150 podman[70612]: 2024-09-10 19:25:19.743571804 +0000 UTC m=+0.531704686 container remove ac8a3d05b787a061ead30d4372436e93cd3989c736a5c6edea8a55174324f992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T19:25:20.289 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.061821435 +0000 UTC m=+0.015845396 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:25:20.289 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.160970433 +0000 UTC m=+0.114994435 container create 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T19:25:20.290 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.034+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:20.290 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.034+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:20.290 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.034+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:20.290 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.034+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:20.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.388358198 +0000 UTC m=+0.342382158 container init 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:25:20.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.391603419 +0000 UTC m=+0.345627384 container start 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:25:20.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.463160226 +0000 UTC m=+0.417184190 container attach 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:25:20.883 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:20 smithi150 podman[70701]: 2024-09-10 19:25:20.564661292 +0000 UTC m=+0.518685257 container died 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T19:25:21.179 INFO:journalctl@ceph.osd.6.smithi150.stdout:Sep 10 19:25:21 smithi150 podman[70724]: 2024-09-10 19:25:21.174963993 +0000 UTC m=+0.604123988 container remove 1633ab5d6434294a10d840ae47ab82510ad12da77b5eb615b5f38a254c371a82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:25:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.992+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.992+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.992+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:21.179 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:20 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:20.992+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:21.195 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T19:25:21.236 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:21.236 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T19:25:21.237 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T19:25:21.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:25:21.555 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:21 smithi150 systemd[1]: Stopping Ceph osd.7 for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:21.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:21 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:21.551+0000 7f059788f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:25:21.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:21 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:21.551+0000 7f059788f640 -1 osd.7 67 *** Got signal Terminated *** 2024-09-10T19:25:21.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:21 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:21.551+0000 7f059788f640 -1 osd.7 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:25:22.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:22 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:22.011+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:22.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:22 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:22.011+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:22.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:22 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:22.011+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:22.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:22 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:22.011+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:23.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:23.045+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:23.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:23.045+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:23.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:23.045+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:23.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:23 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:23.045+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:24.382 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:24.019+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:24.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:24.020+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:24.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:24.020+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:24.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:24 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:24.020+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:25.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:25.067+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:25.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:25.067+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:25.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:25.067+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:25.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:25 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:25.067+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:26.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:26.069+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6803 osd.0 since back 2024-09-10T19:24:27.001530+0000 front 2024-09-10T19:24:27.001523+0000 (oldest deadline 2024-09-10T19:24:52.901460+0000) 2024-09-10T19:25:26.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:26.069+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6807 osd.1 since back 2024-09-10T19:24:34.602305+0000 front 2024-09-10T19:24:34.602417+0000 (oldest deadline 2024-09-10T19:25:04.002765+0000) 2024-09-10T19:25:26.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:26.069+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6811 osd.2 since back 2024-09-10T19:24:47.503726+0000 front 2024-09-10T19:24:47.503708+0000 (oldest deadline 2024-09-10T19:25:12.803589+0000) 2024-09-10T19:25:26.383 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:26 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7[54864]: 2024-09-10T19:25:26.069+0000 7f0594ea9640 -1 osd.7 67 heartbeat_check: no reply from 172.21.15.115:6815 osd.3 since back 2024-09-10T19:24:52.804023+0000 front 2024-09-10T19:24:52.803945+0000 (oldest deadline 2024-09-10T19:25:16.303988+0000) 2024-09-10T19:25:27.137 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:26 smithi150 podman[70803]: 2024-09-10 19:25:26.882460387 +0000 UTC m=+5.420217341 container died 0b390b8be6f9138350e5fe275591df8dc92508026deb47d5df88021ffb0cff5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T19:25:27.454 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:27 smithi150 podman[70803]: 2024-09-10 19:25:27.235623302 +0000 UTC m=+5.773380257 container cleanup 0b390b8be6f9138350e5fe275591df8dc92508026deb47d5df88021ffb0cff5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T19:25:27.455 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:27 smithi150 bash[70803]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7 2024-09-10T19:25:27.882 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:27 smithi150 podman[70819]: 2024-09-10 19:25:27.45354745 +0000 UTC m=+0.565360023 container remove 0b390b8be6f9138350e5fe275591df8dc92508026deb47d5df88021ffb0cff5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T19:25:28.219 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:27 smithi150 podman[70920]: 2024-09-10 19:25:27.805170694 +0000 UTC m=+0.016433059 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:25:28.219 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:27 smithi150 podman[70920]: 2024-09-10 19:25:27.90676503 +0000 UTC m=+0.118027379 container create 19d37f9f6e6473226ca4859b4ffe5cbaba142ba8c39c96571633f82e668765b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:25:28.219 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:28 smithi150 podman[70920]: 2024-09-10 19:25:28.143176556 +0000 UTC m=+0.354438914 container init 19d37f9f6e6473226ca4859b4ffe5cbaba142ba8c39c96571633f82e668765b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:25:28.219 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:28 smithi150 podman[70920]: 2024-09-10 19:25:28.146472029 +0000 UTC m=+0.357734383 container start 19d37f9f6e6473226ca4859b4ffe5cbaba142ba8c39c96571633f82e668765b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T19:25:28.220 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:28 smithi150 podman[70920]: 2024-09-10 19:25:28.218117495 +0000 UTC m=+0.429379845 container attach 19d37f9f6e6473226ca4859b4ffe5cbaba142ba8c39c96571633f82e668765b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:25:28.622 INFO:journalctl@ceph.osd.7.smithi150.stdout:Sep 10 19:25:28 smithi150 podman[70920]: 2024-09-10 19:25:28.335837581 +0000 UTC m=+0.547099931 container died 19d37f9f6e6473226ca4859b4ffe5cbaba142ba8c39c96571633f82e668765b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T19:25:28.911 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T19:25:28.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:28.952 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T19:25:28.952 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T19:25:28.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T19:25:29.234 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:28 smithi115 systemd[1]: Stopping Ceph rgw.foo.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:29.491 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a[65248]: 2024-09-10T19:25:29.231+0000 7f654affd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:25:29.491 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:29 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a[65248]: 2024-09-10T19:25:29.231+0000 7f6553a357c0 -1 shutting down 2024-09-10T19:25:31.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:31.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T19:25:31.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:31 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:31.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused" 2024-09-10T19:25:39.557 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:39 smithi115 bash[82668]: time="2024-09-10T19:25:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:25:39.557 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:39 smithi115 podman[82668]: 2024-09-10 19:25:39.253092879 +0000 UTC m=+10.111504728 container died 2f8dfe5ee32ea272b9b1905cd57874c48993f525ae2acf8a459142d9f122a73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T19:25:39.890 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:39 smithi115 podman[82668]: 2024-09-10 19:25:39.646870135 +0000 UTC m=+10.505281984 container cleanup 2f8dfe5ee32ea272b9b1905cd57874c48993f525ae2acf8a459142d9f122a73b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T19:25:39.890 INFO:journalctl@ceph.rgw.foo.a.smithi115.stdout:Sep 10 19:25:39 smithi115 bash[82668]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T19:25:40.044 DEBUG:teuthology.orchestra.run.smithi115:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T19:25:40.081 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:40.081 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T19:25:40.081 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T19:25:40.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T19:25:40.368 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 systemd[1]: Stopping Ceph prometheus.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:25:40.369 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a[64681]: ts=2024-09-10T19:25:40.357Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:25:40.370 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 podman[71023]: 2024-09-10 19:25:40.367352046 +0000 UTC m=+0.096293743 container died 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:40.979 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 podman[71023]: 2024-09-10 19:25:40.748321432 +0000 UTC m=+0.477263128 container cleanup 4fd52cc5d0e17858d5be9fb8f2e18f64e31d7beb5e8578576e4241fc324ed394 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:40.980 INFO:journalctl@ceph.prometheus.a.smithi150.stdout:Sep 10 19:25:40 smithi150 bash[71023]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:25:41.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo pkill -f 'journalctl -f -n 0 -u ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T19:25:41.167 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:25:41.168 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T19:25:41.168 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm rm-cluster --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:25:41.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:41.628Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T19:25:41.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:41.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi115.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.115:8443: connect: connection refused" 2024-09-10T19:25:41.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:41 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:41.629Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi150.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.150:8443: connect: connection refused" 2024-09-10T19:25:42.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:42 smithi115 systemd[1]: Stopping Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:43.328 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a[73632]: ts=2024-09-10T19:25:43.060Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:25:43.329 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 podman[83119]: 2024-09-10 19:25:43.081004331 +0000 UTC m=+0.124612751 container died 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:43.646 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 podman[83119]: 2024-09-10 19:25:43.393400489 +0000 UTC m=+0.437008916 container cleanup 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:43.646 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 bash[83119]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:25:43.646 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 podman[83132]: 2024-09-10 19:25:43.569482895 +0000 UTC m=+0.482897263 container remove 9a3651e855d6e653d5f8a087ca408244d5f99434b115963c24a58bbd9553bc15 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:43.646 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 podman[83132]: 2024-09-10 19:25:43.645131416 +0000 UTC m=+0.558545800 volume remove 281490dd0165cf712cd7bc1c10876a9cdf87a766ac21147ab513533bc3adfa91 2024-09-10T19:25:43.913 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:25:43.914 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 systemd[1]: Stopped Ceph alertmanager.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:25:43.914 INFO:journalctl@ceph.alertmanager.a.smithi115.stdout:Sep 10 19:25:43 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.124s CPU time. 2024-09-10T19:25:44.292 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 systemd[1]: Stopping Ceph node-exporter.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:44.581 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 podman[83239]: 2024-09-10 19:25:44.291842055 +0000 UTC m=+0.119567863 container died 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:44.582 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 podman[83239]: 2024-09-10 19:25:44.58106583 +0000 UTC m=+0.408791630 container cleanup 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:44.913 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 bash[83239]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T19:25:44.913 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 podman[83251]: 2024-09-10 19:25:44.815702349 +0000 UTC m=+0.523821323 container remove 421ec2b04193c0d99e3244e2459fa9fbac7b43cb9f9b310a96a11c0d903adc85 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:25:44.913 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:25:45.381 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T19:25:45.381 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 systemd[1]: Stopped Ceph node-exporter.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:25:45.381 INFO:journalctl@ceph.node-exporter.a.smithi115.stdout:Sep 10 19:25:44 smithi115 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.492s CPU time. 2024-09-10T19:25:45.587 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm rm-cluster --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:25:47.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:46 smithi150 systemd[1]: Stopping Ceph iscsi.iscsi.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:47.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:47 smithi150 ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a[59926]: debug Shutdown received 2024-09-10T19:25:57.505 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 bash[71396]: time="2024-09-10T19:25:57Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:25:57.506 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 podman[71396]: 2024-09-10 19:25:57.196023187 +0000 UTC m=+10.114794453 container died e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:25:57.877 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 podman[71396]: 2024-09-10 19:25:57.666012115 +0000 UTC m=+10.584783372 container cleanup e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T19:25:57.877 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 bash[71396]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T19:25:58.132 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 podman[71409]: 2024-09-10 19:25:57.875987224 +0000 UTC m=+0.675813239 container remove e3afade4c42a5e34f1e788758b95b731108531ca24c35257185c061d0743170a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:25:58.133 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:57 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T19:25:58.552 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 podman[71462]: 2024-09-10 19:25:58.231024681 +0000 UTC m=+0.138161332 container died b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:25:58.553 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 podman[71462]: 2024-09-10 19:25:58.551955119 +0000 UTC m=+0.459091767 container cleanup b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 bash[71462]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 59760 (conmon) remains running after unit stopped. 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 71476 (podman) remains running after unit stopped. 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: Stopped Ceph iscsi.iscsi.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.603s CPU time. 2024-09-10T19:25:58.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi150.stdout:Sep 10 19:25:58 smithi150 podman[71476]: 2024-09-10 19:25:58.7681554 +0000 UTC m=+0.540001987 container remove b2c010606a9a2bc4ba6d1ef2d952ffa3cd681b8fc7688e965a8b14a5492ec044 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2) 2024-09-10T19:25:59.132 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:58 smithi150 systemd[1]: Stopping Ceph grafana.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:25:59.539 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 podman[71595]: 2024-09-10 19:25:59.240224354 +0000 UTC m=+0.115470202 container died f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.buildah.version=1.32.0, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, com.redhat.component=ubi8-container, release=239.1651231664, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, distribution-scope=public, maintainer=Paul Cuzner , version=8.5, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T19:25:59.539 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 podman[71595]: 2024-09-10 19:25:59.538378144 +0000 UTC m=+0.413624036 container cleanup f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, vcs-type=git, io.buildah.version=1.32.0, maintainer=Paul Cuzner , vendor=Red Hat, Inc., version=8.5, release=239.1651231664, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, io.openshift.expose-services=, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-09-10T19:25:59.882 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 bash[71595]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a 2024-09-10T19:25:59.882 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 podman[71608]: 2024-09-10 19:25:59.773263442 +0000 UTC m=+0.535549970 container remove f00516215b3b9253ab2d957e6d2228a2ee026cb2a67148abfb6428ec8163ffff (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, maintainer=Paul Cuzner , release=239.1651231664, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, com.redhat.component=ubi8-container, distribution-scope=public, name=ubi8, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-10T19:25:59.883 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:26:00.337 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 systemd[1]: Stopping Ceph node-exporter.b for 4f14fa52-6fa8-11ef-bcea-c7b262605968... 2024-09-10T19:26:00.338 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T19:26:00.338 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 systemd[1]: Stopped Ceph grafana.a for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:26:00.338 INFO:journalctl@ceph.grafana.a.smithi150.stdout:Sep 10 19:25:59 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.806s CPU time. 2024-09-10T19:26:00.594 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 podman[71716]: 2024-09-10 19:26:00.336785605 +0000 UTC m=+0.102383864 container died b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:26:00.882 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 podman[71716]: 2024-09-10 19:26:00.709224866 +0000 UTC m=+0.474823116 container cleanup b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:26:00.882 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 bash[71716]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T19:26:01.274 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 podman[71728]: 2024-09-10 19:26:00.993610741 +0000 UTC m=+0.657029849 container remove b67496bf48c59e2f152b13efee0b08dbf9d276fbce044b92354bcaa2a35e119b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:26:01.274 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:00 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:26:01.274 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:01 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T19:26:01.274 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:01 smithi150 systemd[1]: Stopped Ceph node-exporter.b for 4f14fa52-6fa8-11ef-bcea-c7b262605968. 2024-09-10T19:26:01.274 INFO:journalctl@ceph.node-exporter.b.smithi150.stdout:Sep 10 19:26:01 smithi150 systemd[1]: ceph-4f14fa52-6fa8-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.421s CPU time. 2024-09-10T19:26:01.766 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:26:01.798 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:26:01.825 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T19:26:01.827 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi115/crash 2024-09-10T19:26:01.827 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:26:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:tar: /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:26:01.925 INFO:teuthology.orchestra.run.smithi115.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:26:01.927 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi150/crash 2024-09-10T19:26:01.927 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:26:01.967 INFO:teuthology.orchestra.run.smithi150.stderr:tar: /var/lib/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:26:01.967 INFO:teuthology.orchestra.run.smithi150.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:26:01.968 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T19:26:01.968 DEBUG:teuthology.orchestra.run.smithi115:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-10T19:26:01.994 INFO:teuthology.orchestra.run.smithi115.stderr:grep: /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T19:26:01.996 INFO:tasks.cephadm:Compressing logs... 2024-09-10T19:26:01.996 DEBUG:teuthology.orchestra.run.smithi115:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:26:02.038 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:26:02.060 INFO:teuthology.orchestra.run.smithi115.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T19:26:02.060 INFO:teuthology.orchestra.run.smithi115.stderr:: No such file or directory 2024-09-10T19:26:02.060 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T19:26:02.061 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:26:02.062 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T19:26:02.062 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T19:26:02.065 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T19:26:02.065 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T19:26:02.065 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T19:26:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T19:26:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:: No such file or directory 2024-09-10T19:26:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T19:26:02.067 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/tcmu-runner.log: 63.5% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:26:02.068 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:26:02.072 INFO:teuthology.orchestra.run.smithi150.stderr: 91.2% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:26:02.073 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.y.log: 91.2% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:26:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T19:26:02.074 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T19:26:02.075 INFO:teuthology.orchestra.run.smithi115.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:26:02.090 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T19:26:02.091 INFO:teuthology.orchestra.run.smithi115.stderr:/var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:26:02.269 INFO:teuthology.orchestra.run.smithi115.stderr: 88.3% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T19:26:02.351 INFO:teuthology.orchestra.run.smithi150.stderr: 91.6% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T19:26:02.370 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T19:26:02.410 INFO:teuthology.orchestra.run.smithi115.stderr: 91.6% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T19:26:02.429 INFO:teuthology.orchestra.run.smithi150.stderr: 92.8% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T19:26:02.469 INFO:teuthology.orchestra.run.smithi150.stderr: 93.0% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T19:26:02.497 INFO:teuthology.orchestra.run.smithi115.stderr: 92.7% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T19:26:02.500 INFO:teuthology.orchestra.run.smithi150.stderr: 93.1% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T19:26:02.501 INFO:teuthology.orchestra.run.smithi150.stderr: 2024-09-10T19:26:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m0.449s 2024-09-10T19:26:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m1.779s 2024-09-10T19:26:02.502 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.079s 2024-09-10T19:26:02.545 INFO:teuthology.orchestra.run.smithi115.stderr: 92.8% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T19:26:02.573 INFO:teuthology.orchestra.run.smithi115.stderr: 92.9% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T19:26:02.644 INFO:teuthology.orchestra.run.smithi115.stderr: 93.1% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T19:26:02.666 INFO:teuthology.orchestra.run.smithi115.stderr: 90.7% -- replaced with /var/log/ceph/4f14fa52-6fa8-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T19:26:02.668 INFO:teuthology.orchestra.run.smithi115.stderr: 2024-09-10T19:26:02.668 INFO:teuthology.orchestra.run.smithi115.stderr:real 0m0.618s 2024-09-10T19:26:02.668 INFO:teuthology.orchestra.run.smithi115.stderr:user 0m3.035s 2024-09-10T19:26:02.668 INFO:teuthology.orchestra.run.smithi115.stderr:sys 0m0.111s 2024-09-10T19:26:02.668 INFO:tasks.cephadm:Archiving logs... 2024-09-10T19:26:02.669 DEBUG:teuthology.misc:Transferring archived files from smithi115:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi115/log 2024-09-10T19:26:02.670 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:26:03.112 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi150/log 2024-09-10T19:26:03.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:26:03.386 INFO:tasks.cephadm:Removing cluster... 2024-09-10T19:26:03.386 DEBUG:teuthology.orchestra.run.smithi115:> sudo cephadm rm-cluster --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 --force 2024-09-10T19:26:03.844 DEBUG:teuthology.orchestra.run.smithi150:> sudo cephadm rm-cluster --fsid 4f14fa52-6fa8-11ef-bcea-c7b262605968 --force 2024-09-10T19:26:04.314 INFO:tasks.cephadm:Teardown complete 2024-09-10T19:26:04.314 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T19:26:04.325 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T19:26:04.325 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:26:04.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:26:04.401 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T19:26:04.410 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T19:26:04.410 DEBUG:teuthology.orchestra.run.smithi115:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:26:04.413 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:26:04.425 INFO:teuthology.orchestra.run.smithi115.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:26:04.427 INFO:teuthology.orchestra.run.smithi150.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:26:04.429 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:26:04.429 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2024-09-10T19:26:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 -386us[ -371us] +/- 40ms 2024-09-10T19:26:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 +1508us[+1508us] +/- 59ms 2024-09-10T19:26:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 -1362us[-1346us] +/- 85ms 2024-09-10T19:26:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:26:04.431 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:26:04.431 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2024-09-10T19:26:04.432 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 -370us[ -342us] +/- 40ms 2024-09-10T19:26:04.432 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 +1507us[+1535us] +/- 59ms 2024-09-10T19:26:04.432 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 -1360us[-1331us] +/- 85ms 2024-09-10T19:26:04.432 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:26:04.432 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T19:26:04.440 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T19:26:04.441 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T19:26:04.450 DEBUG:teuthology.orchestra.run.smithi115:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:26:04.542 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:26:04.612 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:26:04.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:26:04.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:26:04.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:26:04.670 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T19:26:04.679 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T19:26:04.690 INFO:teuthology.task.internal:Duration was 1321.472476 seconds 2024-09-10T19:26:04.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T19:26:04.699 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T19:26:04.700 DEBUG:teuthology.orchestra.run.smithi115:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:26:04.702 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:26:04.738 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:26:04.748 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:26:04.950 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T19:26:04.950 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi115.front.sepia.ceph.com 2024-09-10T19:26:04.950 DEBUG:teuthology.orchestra.run.smithi115:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:26:05.012 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2024-09-10T19:26:05.012 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:26:05.046 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T19:26:05.046 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:26:05.054 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:26:05.113 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T19:26:05.113 DEBUG:teuthology.orchestra.run.smithi115:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:26:05.121 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:26:05.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T19:26:05.411 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T19:26:05.411 DEBUG:teuthology.orchestra.run.smithi115:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:26:05.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:26:05.460 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T19:26:05.470 DEBUG:teuthology.orchestra.run.smithi115:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:26:05.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:26:05.499 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = core 2024-09-10T19:26:05.524 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2024-09-10T19:26:05.548 DEBUG:teuthology.orchestra.run.smithi115:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:26:05.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:26:05.587 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:26:05.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:26:05.602 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T19:26:05.612 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T19:26:05.612 DEBUG:teuthology.misc:Transferring archived files from smithi115:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi115 2024-09-10T19:26:05.613 DEBUG:teuthology.orchestra.run.smithi115:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:26:05.667 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900288/remote/smithi150 2024-09-10T19:26:05.667 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:26:05.704 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T19:26:05.705 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:26:05.707 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:26:05.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T19:26:05.763 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T19:26:05.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T19:26:05.771 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T19:26:05.771 DEBUG:teuthology.orchestra.run.smithi115:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:26:05.773 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:26:05.786 INFO:teuthology.orchestra.run.smithi115.stdout: 265683 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:26 /home/ubuntu/cephtest 2024-09-10T19:26:05.807 INFO:teuthology.orchestra.run.smithi150.stdout: 265805 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:26 /home/ubuntu/cephtest 2024-09-10T19:26:05.808 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T19:26:05.818 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T19:26:05.838 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} duration: 1321.4724757671356 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T19:26:05.838 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T19:26:05.912 INFO:teuthology.run:pass